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ABSTRACT 

Three  discrete  reliability  growth  models  using  fractional  failure  reduction,  referred 
to  as  failure  discounting,  were  developed  to  estimate  changing  system  reliability.  Each 
of  the  models  is  designed  for  use  when  testing  is  performed  until  a  fixed  number  of 
failures  have  been  observed  and  attribute  data,  success  or  failure,  is  available  for  each 
trial.  The  first  reliability  growth  model  applies  failure  discounting  to  the  maximum 
likelihood  estimate  for  a  proportion.  The  second  and  third  models  use  a  modification  of 
an  exponential  reliability  estimate  employing  linear  regression  and  a  weighted  average 
technique  respectively  along  with  failure  discounting  to  track  changing  reliability.  Two 
failure  discounting  methods  were  used  with  each  reliability  growth  model.  The  first 
method  reduces  past  failures  by  a  fixed  fraction  at  a  fixed  interval.  The  second  method 
uses  the  upper  confidence  bound  for  the  reoccurrence  of  each  failure  cause  as  the 
discounted  failure  value.  The  performance  of  the  reliability  growth  models  with 
varying  reliability  growth  patterns  was  evaluated  with  a  Monte-Carlo  simulation. 
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I.  INTRODUCTION 

Accurately  determining  the  reliability  of  a  newly  designed  piece  of  hardware  is  an 
important  part  of  material  acquisition  in  both  the  military  and  civilian  sectors. 
Significant  amounts  of  both  time  and  money  are  spent  testing  new  systems  to 
determine  if  they  meet  specified  reliability  requirements.  Since  most  organizations  are 
under  constant  pressure  to  conserve  both  time  and  funds,  it  is  important  to  develop 
methods  of  estimating  system  reliability  at  minimum  cost.  When  conventional 
estimators  for  system  reliability  are  used,  verifying  high  reliability  requires  a  great  deal 
of  test  data  costing  both  time  and  money.  This  problem  is  compounded  by 
technological  advances  which  produce  more  complex  and  highly  reliable  hardware  at 
an  ever  increasing  cost.  Developing  new  and  more  accurate  methods  of  estimating 
reliability  that  reduce  required  testing  could  result  in  significant  savings  of  resources. 

Early  versions  of  newly  designed  equipment  often  exhibit  low  reliability.  As 
engineering  designs  are  tested  and  improved  the  system  reliability  usually  grows 
rapidly.  This  process  continues  throughout  development  phases  and  into  field  use.  By 
the  time  a  product  design  reaches  its  final  configuration,  a  large  amount  of  test  data  is 
usually  available  from  tests  performed  on  previous  configurations,  subcomponents  and 
parts.  Since  conventional  estimators  assume  a  constant  reliability,  it  is  necessary  to 
wait  until  the  finished  piece  of  hardware  is  available  to  begin  reliability  verification. 
This  practice  is  not  an  efficient  use  of  constrained  resources  because  results  from 
previous  testing  are  discarded.  If  all  the  data  available  could  be  used  to  estimate  final 
configuration  reliability,  the  lead  time  and  cost  of  the  finished  product  could  be 
reduced.  Reliability  growth  models  serve  this  purpose.  They  use  data  from  testing  at 
all  phases  of  development  to  establish  a  reliability  growth  pattern.  This  growth  pattern 
can  then  be  used  to  provide  current  estimates  of  system  reliability  with  less  testing  than 
standard  testing  programs. 

Reliability  growth  models  can  be  classified  into  two  categories,  discrete  and 
continuous  models.  This  paper  will  address  discrete  reliability  growth  models  that 
employ  failure  discounting.  These  growth  models  are  used  to  determine  reliability  for 
systems  where  test  results  can  only  be  stated  in  terms  of  success  or  failure,  ie;  attribute 
data.  Failure  discounting  allows  the  user  to  fractionally  discount  past  failures  when 
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resultant  corrections  to  equipment  are  verified  through  subsequent  testing,  as  actual 
corrections  to  the  cause  of  failure.  The  use  of  failure  discounting  makes  it  possible  for 
previously  linear  estimators  to  track  a  non-linear  growth  pattern. 

The  objective  of  this  paper  is  to  develop  and  evaluate  the  performance  of  discrete 
reliability  growth  models  using  failure  discounting.  Three  different  reliability  growth 
models  are  proposed  and  evaluated  with  two  different  methods  for  discounting  past 
failures.  Monte-Carlo  simulation  is  used  to  evaluate  each  of  the  three  growth  models 
under  both  discounting  methods.  The  simulation  uses  actual  reliabilities  to  generate  a 
random  growth  pattern.  From  this  growth  pattern,  test  data  is  then  generated  for  the 
known  system  reliability  at  each  testing  phase.  Each  of  the  reliability  growth  models 
can  then  be  applied  to  the  test  data  producing  an  estimated  growth  pattern.  The 
performance  of  each  reliability  growth  model  is  evaluated  by  comparing  the  estimated 
growth  pattern  with  the  actual  reliability  in  each  phase. 

The  following  chapter  will  address  each  of  the  three  reliability  growth  models 
used  and  the  two  methods  of  failure  discounting  employed.  Chapter  III  contains  a 
discussion  of  the  computer  simulation  used  to  evaluate  each  of  the  models.  Chapter  IV 
presents  the  results  of  the  evaluation  and  Chapter  V  summarizes  the  study  stating 
conclusions  and  recomendations  for  future  study. 
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II.  DISCRETE  RELIABILITY  GROWTH  MODELS 

A.       BACKGROUND 

Three  different  reliability  growth  models  and  two  different  techniques  for 
fractionally  discounting  past  failures  are  discussed  in  this  chapter.  The  reliability 
growth  models  can  be  divided  into  two  categories.  Models  using  a  modification  of  the 
maximum  likelihood  estimate,  MLE,  for  a  proportion,  referred  to  as  the  MLE  With 
Discounting  Model,  a  model  referred  to  as  the  Weighted  Average  Model  and  an 
Exponential  Regression  Model.  All  of  the  models  may  be  applied  with  or  without 
failure  discounting  and  are  independent  of  the  discounting  method. 

Two  different  methods  for  fractionally  discounting  past  failures  are  also 
discussed.  The  first  method  is  referred  to  as  the  Standard  or  Straight  Percent 
Discounting  Method.  The  second  was  proposed  by  David  K.  Lloyd  and  will  be 
referred  to  as  the  Lloyd  Method.  Before  the  specific  details  of  each  growth  model  and 
discounting  method  are  discussed,  several  terms  need  clarification. 

Reliability,  R,  as  referred  to  in  this  study  is  defined  as  the  probability  of  a  system 
successfully  completing  a  single  trial  or  operation  cycle,  Equation  2.1  .  R  will  be  used 
to  denote  an  estimate  of  the  actual  reliability  R. 

R  =  Pr  (  SINGLE  TRIAL  SUCCESS  )  (eqn  2.1) 

Discrete  reliability  growth  models  are  normally  used  for  trials  which  can  only  be 
evaluated  in  terms  of  a  binary  variable,  success  or  failure,  i.e.  attribute  data.  Exactly 
what  constitutes  a  trial  is  normally  defined  by  the  test  agency  but  could  include,  for 
example,  a  single  rocket  engine  ignition  attempt  or  a  single  test  firing  of  a  missile  up  to 
and  including  target  destruction.  To  use  the  reliability  growth  models  discussed  here, 
trials  must  be  defined  such  that  they  are  discrete  and  can  be  evaluated  as  a  success  or  a 
failure. 

A  testing  phase  is  defined  as  a  collection  of  trials,  one  or  more,  for  which  the 
configuration  of  the  system  being  tested  is  unchanged.  This  implies  that  the  actual 
system  reliability  remains  constant  throughout  a  single  phase.  If  a  test-fix-test 
methodology  is  employed,  design  changes  or  "fixes"  are  implemented  after  each  failure. 


12 


In  this  case,  a  phase  will  consist  of  one  failed  trial  and  all  the  successful  trials  leading 
up  to  the  single  failure.  If  on  the  other  hand  a  test-fmd-test  methodology  is  used,  a 
single  phase  may  include  numerous  failures.  Under  both  methodologies,  a  phase  is  the 
collection  of  all  trials  between  configuration  changes.   [Ref.  1] 

Because  the  underlying  reliability  of  the  system  may  be  different  from  phase  to 
phase,  a  new  estimate  of  reliability  must  be  computed  for  each  and  every  phase.  As 
such,  Rk  .denoting  system  reliability  and  Rk  denoting  the  estimate  of  system  reliability 
in  phase  k,  will  be  used  throughout  the  paper. 

Annotating  the  cause  of  each  failure  is  critical  to  all  discounting  methods 
considered  in  this  study.  How  the  failure  cause  is  defined,  is  dependent  upon  the 
judgement  of  the  model's  user  and  the  type  of  system  to  which  the  model  is  applied.  If 
the  model  is  used  to  determine  the  reliability  of  a  single  component,  failure  cause  refers 
to  the  reason  for  failure  such  as  a  broken  cam-shaft  or  fuel  stoppage.  If  the  model  is 
applied  to  determine  the  reliability  of  a  system  of  components,  the  failure  cause  may 
refer  to  the  failure  of  a  sub-component  or  the  specific  reason  for  failure.  To  apply  the 
discounting  methods  discussed  in  the  paper,  the  model  user  must  determine  if  two 
observed  failures  are  due  to  the  same  cause.  It  will  not  be  necessary,  however,  to 
enumerate  all  possible  failure  causes.  The  level  of  detail  at  which  failure  causes  are 
defined  is  up  to  the  user  and  should  be  determined  before  any  testing  is  begun.  The 
discounting  methods  studied  will  be  discussed  in  the  next  section. 

B.       DISCOUNTING  METHODOLOGY 

Weaknesses  in  a  system  design  become  known  as  they  cause  failures  during 
testing.  As  weaknesses  are  identified  and  corrective  "fixes"  are  implemented  the 
probability  of  a  repeat  failure  due  to  the  corrected  cause  should  be  reduced  and  system 
reliability  improved.  This  concept  forms  the  basis  for  failure  discounting.  Once  a 
system  weakness  has  been  corrected  and  the  improvement  validated  through  further 
testing,  fractionally  discounting  the  past  failure  is  a  method  of  reflecting  the  improved 
system  reliability  in  the  previously  collected  data.  When  using  standard  reliability 
estimators  such  as  the  maximum  likelihood  estimate,  Equation  2.2 


Number  of  Successful  Trials 

R  =    (eqn  2.2) 

Total  Number  of  Trials 
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only  the  data  from  the  phase  of  interest  can  be  used  to  determine  system  reliability 
because  system  reliability  is  different  from  phase  to  phase.  If  the  proper  discounting 
methodology  can  be  developed,  failure  data  from  all  past  phases  could  be  discounted 
and  made  compatible  with  the  data  from  the  current  phase.  Increasing  the  amount  of 
data  available  for  computing  the  estimate  of  current  system  reliability  will  increase  the 
accuracy  of  the  estimate  and  reduce  the  amount  of  testing  necessary  to  verify  system 
reliability. 

Of  equal  importance  to  a  method  for  discounting  past  failures  is  a  penalty  system 
that  will  restore  all  or  some  portion  of  a  previously  discounted  failure  if  its  failure 
cause  should  reoccur.  If  a  previously  implemented  "fix"  proves  not  to  have  corrected  a 
design  weakness,  the  discounts  applied  to  its  failures  will  diminished  -  erased  reversing 
the  effect  of  discounting.  A  properly  designed  penalty  factor  will  make  the  discounting 
method  self-correcting. 

In  order  for  a  discounting  method  to  be  useful,  it  must  be  flexible  enough  to 
handle  many  varied  reliability  growth  patterns.  To  achieve  this  flexibility,  discounting 
methods  with  one  or  more  input  parameters  are  proposed.  Regardless  which 
discounting  method  is  used,  experience  and  good  engineering  judgement  will  be 
necessary  to  choose  input  parameters  that  are  compatible  with-  both  the  reliability 
estimator  used  and  the  proposed  test  plan.  If  improper  parameters  are  applied,  the 
model  may  consistently  over  or  underestimate  system  reliability.  The  following 
sections  describe  the  specific  steps  for  applying  two  failure  discounting  methods 
including  examples  of  their  use. 

1.  Straight  Percent  Discounting  Method 

The  Straight  Percent  Discounting  Method  removes  a  fixed  fraction  of  a  failure 
each  time  a  predetermined  number  of  trials  are  observed  without  reoccurrence  of  the 
failure  cause.  Before  a  failure  is  discounted,  the  design  "fix"  correcting  the  failure  cause 
should  be  verified  by  further  testing.  This  concept  forms  the  basis  for  the  discounting 
method.  The  fraction  of  a  failure  removed,  F,  is  referred  to  as  the  discount  fraction. 
After  the  initial  occurrence  of  a  particular  failure  cause,  any  trial  in  which  the  failure 
cause  is  not  repeated  is  considered  a  success.  The  number  of  successful  trials  between 
application  of  the  discount  fraction  is  referred  to  as  the  discount  interval,  N.  Both  N 
and  F  are  input  parameters  specified  by  the  user.  They  remain  constant  throughout 
failure  discounting  computations.  To  use  the  Straight  Percent  Discounting  Method,  it 
is  necessary  to  record  the  number  of  successful  trials  since  the  last  occurrence  of  each 
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observed  failure  cause,  referred  to  as  M.  Failure  discounting  is  performed  at  the  end  of 
each  testing  phase  using  the  values  for  N,  F  and  M. 

The  mathematical  representation  of  the  straight  percent  discounting  method  is 
shown  in  Equation  2.3 

ADJUSTED  FAILURE  =  (1  -  F)INT^M/N)  (eqn  2.3) 

F  =  Fraction  of  a  Failure  to  be  Removed 

M  =  Number  of  Successful  Trials  Since  Last  Failure  for  the  Failure  Cause 
N  =  The  Discount  Interval  Expressed  in  the  Number  of  Successful  Trials 

INT  is  a  function  which  returns  the  truncated  integer  portion  of  its  argument.  M  is  set 
to  zero  at  each  occurrence  or  reoccurrence  of  a  particular  failure  cause.  This  causes  the 
value  for  adjusted  failure  to  be  set  at  1.0  erasing  any  previously  applied  discounting. 
This  also  functions  as  a  penalty  factor  allowing  the  model  to  correct  itself  should  past 
discounting  prove  unwarranted.  By  altering  the  choice  of  N  and  F  the  amount  of 
discounting  applied  can  be  varied  from  none  to  total  loss  of  a  failure.  This  allows  the 
method  to  be  applied  in  situations  verying  from  little  or  no  reliability  growth  to  rapid 
reliability  growth. 

An  example  will  help  clarify  the  use  of  the  method.  Consider  the  test  results 
included  in  Table  1.  The  data  displayed  in  the  table  is  from  a  test-fix-test  program  in 
time  order  sequence.  A  design  fix  was  applied  to  the  equipment  after  each  failure.  As 
such,  the  actual  system  reliability  does  not  remain  constant.  The  outcome  of  each  trial 
was  generated  from  the  geometric  distribution  using  the  actual  reliability  listed  in  the 
table.  The  cause  for  each  failure  was  generated  using  a  stochastic  process  and  has 
been  coded  using  letters.  No  more  information  concerning  failure  cause  is  required 
because  the  data  are  used  to  determine  if  the  current  failure  cause  has  occurred 
previously. 

To.  apply  Straight  Percent  Discounting  the  two  input  parameters  must  be 
determined.  For  the  example 

N  =  3 

F    =  .25 
will  be  used.  The  Straight  Percent  Discounting  Method  is  applied  sequentially,  failure 
by  failure,  recomputing  M,  the  number  of  successful  trials  since  last  failure,  for  each 
failure  cause  as  the  next  data  entry  is  included.  At  the  end  of  each  testing  phase,  the 
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TABLE  1 

SAMPLE  TEST  RESULTS 

TRIAL 

FAILURE 

FAILURE  CAUSE 

NUMBER 

NUMBER 

A 

B 

c 

D 

RELIABILITY 

1 

S 

s 

s 

s 

.358 

2 

1 

F 

S 

s 

s 

.358 

3 

2 

S 

F 

s 

s 

.412 

4 

3 

S 

S 

F 

s 

.518 

5 

4 

s 

S 

s 

F 

.596 

6 

s 

s 

s 

S 

.627 

7 

5 

s 

s 

F 

s 

.627 

8 

s 

s 

s 

s 

.721 

9 

.  s 

s 

s 

s 

.721 

10 

6 

s 

F 

s 

s 

.721 

11 

s 

s 

s 

s 

.780 

12 

s 

s 

s 

s 

.780 

13 

s 

s 

s 

s 

.780 

14 

s 

s 

s 

s 

.780 

15 

s 

s 

s 

s 

.780 

16 

s 

s 

s 

s 

.780 

17 

s 

s 

s 

s 

.780 

18 

7 

F 

s 

s 

s 

.780 

S 

DENOTES  SUCCESS 

F: 

DENOTES  FAILURE 

value  of  M,  along  with  the  input  parameters  F  and  N,  is  used  in  Equation  2.3  to 
determine  the  adjusted  failure  value.   This  procedure  is  applied  to  the  data  in  Table  1. 

When  the  sixth  failure  is  included,  all  failures  due  to  Causes  A,  C  or  D  have 
values  for  M  greater  than  or  equal  to  three  causing  the  discount  fraction  to  be  applied. 
Computation  of  the  adjusted  failure  values  for  Cause  A  is  illustrated  in  the  following 
calculations. 
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ADJUSTED  FAILURE  =  (1  -  .25)INT(8>'3) 

=  (  -75  )2 
=  .5625 

Failure  two  and  failure  six  were  both  due  to  Cause  B  which  resets  their  corresponding 
M  values  to  zero  and  their  adjusted  failure  value  to  unity.  The  results  for  all  failures  to 
date  are  shown  below. 


FAILURE 

FAILURE 

ADJUSTED 

NUMBER 

M 

CAUSE 

FAILURE 

1 

8 

CAUSE  A 

.5625 

2 

0 

CAUSE  B 

1.0000 

3 

CAUSE  C 

.7500 

4 

5 

CAUSE  D 

.7500 

5 

-% 
j 

CAUSE  C 

.7500 

6 

0 

CAUSE  B 

1.0000 

Finally,  failure  seven  is  included  and  the  failure  discounting  computations  are 
completed.  The  number  of  trials,  eight,  causes  further  discounting  for  all  previous 
failures  except  for  the  fust  failure  which  is  also  due  to  Cause  A.  The  fmal  results  of 
applying  the  straight  percent  discount  method  to  the  data  in  Table  1  appears  in  Table 
2.  The  values  in  the  adjusted  failure  column  are  the  current  values  of  each  fractional 
failure  after  trial  number  eighteen  has  been  included.  For  example,  the  two  adjusted 
failure  values  due  to  Cause  B  have  recorded  eight  successfull  trials  since  the  last  failure 
due  to  Cause  B  and  INT  (8/3)  =  2.  Therefore,  the  adjusted  failure  value  equals 
(.75)2  =  .5625.  The  remaining  adjusted  failures  are  computed  in  the  same  manner 
yielding  the  fractional  failure  values  after- eighteen  trials  and  seven  failures. 
2.  Lloyd  Failure  Discounting  Method 

The  underlying  premise  of  the  Lloyd  Discounting  Method  states  that  the 
fraction  of  a  failure  removed  by  the  discounting  method  should  not  be  arbitrarily 
chosen.  Some  statistical  basis  should  be  used  to  determine  how  much,  if  any,  a  past 
failure  should  be  discounted.  Lloyd  proposes  using  the  upper  confidence  limit  for  the 
probability  that  a  failure  cause  will  reoccur  as  the  discounted  value  of  its  corresponding 
failure.  This  technique  allows  the  user  to  control  the  amount  failures  are  discounted  by 
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TABLE  2 

STRAIGHT  PERCENT  DISCOUNTING 
OF  EXAMPLE  DATA  (  TABLE  1  ) 

TRIAL 

FAILURE 

FAILURE  CAUSE 

ADJUSTED 

NUMBER 

NUMBER 

A 

B 

c 

D 

FAILURE 

1 

S 

S 

s 

s 

2 

1 

F 

s 

s 

s 

1.0000 

3 

2 

S 

F 

s 

s 

.5625 

4 

3 

s 

S 

F 

s 

.4219 

5 

4 

s 

S 

s 

F 

.3164 

6 

s 

S 

s 

s 

7 

5 

s 

s 

F 

s 

.4219 

8 

s 

s 

S 

s 

9 

s 

s 

S 

s 

10 

6 

s 

F 

S 

s 

.5625 

11 

s 

s 

S 

s 

12 

s 

S 

s 

s 

13 

s 

s 

s 

s 

14 

s 

s 

s 

s 

15 

s 

s 

s 

s 

16 

s 

s 

s 

s 

17 

s 

s 

s 

s 

18 

7 

F 

s 

s 

s 

1.0000 

S 

:  DENOTES  SUCCESS 

F  : 

DENOTES  FAILURE 

specifying  the  level  of  confidence  for  the  confidence  bound.  As  such,  a  single  input 
parameter,  the  confidence  level  y,  is  required  to  perform  failure  discounting.   [Ref.  2] 
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The  discounting  equation  is  derived  from  the  upper  confidence  limit  for  the 
probability  of  failure  due  to  a  particular  failure  cause.  This  confidence  interval  is  based 
on  the  number  of  successful  trials  recorded  since  implementation  of  the  "fix"  for  the 
failure  cause.  Assuming  the  "fix"  is  implemented  immediately  after  failure,  this  will  be 
equivalent  to  the  number  of  successful  trials  since  last  failure,  previously  defined  as  M. 

Q  =  PROBABILITY  OF  FAILURE  IN  A  SINGLE  TRIAL 

=  1  -  R 
y  =  FRACTIONAL  CONFIDENCE  LEVEL 

M  =  NUMBER  OF  SUCCESSFUL  TRIALS  SINCE  LAST  FAILURE 
£L/y)  =  LOWER  y%  CONFIDENCE  LIMIT  FOR  RELIABILITY 

%y)  =  (l-Y)1/M 

Y  =  Pr  [  R  >  RL(y)  ] 

=  Pr[l-R  <   l-£L(y)] 
=  Pr[Q  <;  1-£L(Y)] 

.".  UPPER  CONFIDENCE  LIMIT  for  Q    -  1  -  (  1  -  7  )l'M      (eqn  2.4) 

ADJUSTED  FAILURE  =  1  -  (  1  -  7  )1/M  for  M  >  0         (eqn  2.5) 

=  1.0  for  M  =  0 

The  failure  discounting  equation  is  derived  in  Equations  2.4  and  2.5.  Because  the 
derived  equation  for  adjusted  failures  is  not  defined  for  M  equal  zero,  the  adjusted 
failure  value  is  set  equal  to  one  where  M  equals  zero.  This  is  the  logical  choice  since  no 
discounting  should  be  applied  to  a  failure  cause  that  is  the  current  cause  of  system 
failure.  This  also  has  the  advantage  of  providing  a  penalty  factor  that  will  restore 
fractional  failures  to  unity  if  a  failure  cause  reoccurs. 

To  apply  the  Lloyd  discounting  method,  the  confidence  level.  7,  must  be 
specified.  The  method  is  then  applied  sequentially,  failure  by  failure,  recomputing  M, 
the  number  of  successful  trials  since  last  failure,  for  each  observed  failure  cause  as  the 
data  from  each  trial  is  included.  At  each  step,  M  is  used  in  Equation  2.5  to  compute  a 
new  adjusted  failure  value  for  each  observed  failure. 
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An  example  will  help  clarify  the  use  of  the  Lloyd  method.  Applying  the 
method  to  the  sample  data  in  Table  1  with  y  equal  to  .99  yields  the  following  results 
after  the  second  failure  is  included. 


FAILURE 

FAILURE 

ADJUSTED 

NUMBER 

M 

CAUSE 

FAILURE 

1 

1 

CAUSE  A 

.9900 

2 

0 

CAUSE  B 

1.0000 

Including  the  data  for  the  third  failure,  due  to  failure  Cause  C,  both  Cause  A  and  B  are 
further  discounted.   The  adjusted  failure  value  for  Cause  A  is  computed  below. 


M  =  2 

ADJUSTED  FAILURE  =   1  -  (  1  -  .99  ) 

=  1  -  V  -01 
=  1  -  .1 


1/2 


The  complete  results  are  shown  below. 


FAILURE 

FAILURE 

ADJUSTED 

NUMBER 

M 

CAUSE 

FAILURE 

1 

2 

CAUSE  A 

.9000 

2 

1 

CAUSE  B 

•  .9900 

3 

0 

CAUSE  C 

1.0000 

The  process  continues  sequentially,  adding  the  next  failure,  recomputing  M,  and 
computing  the  adjusted  failure  values  for  each  previous  failure.  The  final  results  after 
all  seven  failures  are  included  is  shown  in  Table  3.  Note,  as  a  failure  cause  reoccurs  its 
corresponding  value  for  M  is  set  to  zero  forcing  the  adjusted  failure  value  to  unity. 

This  concludes  the  description  of  the  two  discounting  methods  covered  in  the 
study.  The  following  section  will  describe  the  three  discrete  reliability  growth  models. 
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TABLE  3 

LLOYD  DISCOUNTING 
OF  EXAMPLE  DATA  (  TABLE  1 

) 

TRIAL 

FAILURE 

FAILURE  CAUSE 

ADJUSTED 

NUMBER 

NUMBER 

A 

BCD 

FAILURE 

1 

S 

S 

s 

s 

2 

1 

F 

S 

s 

s 

1.0000 

3 

2 

S 

F 

s 

s 

.4377 

4 

3 

S 

S 

F 

s 

.3421 

5 

4 

s 

s 

s 

F 

.2983 

6 

s 

s 

s 

S 

7 

5 

s 

s 

F 

S 

.3421 

8 

s 

s 

s 

S 

9 

s 

s 

S 

s 

10 

6 

s 

F 

S 

s 

.4377 

11 

s 

S 

s 

s 

12 

s 

S 

s 

s 

13 

s 

S 

s 

s 

14 

s 

S 

s 

s 

15 

s 

S 

s 

s 

16 

s 

s 

s 

s 

17 

s 

s 

s 

s 

18 

7 

F 

s 

s 

s 

1.0000 

S 

:  DENOTES  SUCCESS 

F: 

DENOTES  FAILURE 

C.       RELIABILITY  GROWTH  MODELS  DESCRIPTION 

Of  particular  interest  in  this  study  are  test  plans  which  terminate  each  test  phase 
with  system  failure.  Such  test  plans  are  known  as  testing  to  a  fixed  number  of  failures. 
Only  one  of  the  three  reliability  growth  models  proposed  can  be  used  for  test  scenarios 
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which  terminate  testing  after  a  fixed  number  of  trials.  Because  of  this  limitation,  only 
test  plans  which  test  to  a  fixed  number  of  failures  are  considered  in  this  study. 

Three  discrete  reliability  growth  models  are  presented  in  this  section,  each  of 
which  uses  a  different  technique  to  track  changing  system  reliability.  The  first  model, 
Maximum  Likelihood  Estimate  With  Discounting,  is  derived  from  the  conventional 
maximum  likelihood  estimate  of  reliability.  Failure  discounting  gives  the  model  the 
ability  to  track  changing  reliability.  Two  additional  models  derived  from  an 
exponential  single  phase  reliability  estimate  referred  to  as  the  Weighted  Average  Model 
and  the  Exponential  Regression  Model  are  also  discussed.  The  MLE  With 
Discounting  model  will  be  presented  in  the  next  section. 

1.  Maximum  Likelihood  Estimate  With  Failure  Discounting 

The  most   commonly  used  estimate   of  system  reliability  is  the  maximum 
likelihood  estimate  Equation  2.2  restated  below. 


Number  of  Successful  Trials 

R  =    (eqn  2.2) 

Total  Number  of  Trials 


Where  the  number  of  successful  trials  equals  the  total  number  of  trials  minus  the 
number  of  observed  failures.  Because  all  testing  considered  is  conducted  to  a  fixed 

k 
NO.  SUCCESSFUL  TRIALS  =    t    (  T.  -  1  )       (eqn  2.6) 

number  of  failures,  the  number  of  successful  trials  simplifies  to  Equation  2.6.  Where  T. 
equals  the  number  of  trials  following  the  (i—  1)  st  failure  including  the  i  failure  and 
k  equals  the  total  number  of  failures  in  the  test  program  "to  date. 

Two  problems  arise  when  using  Equation  2.2  to  estimate  reliability  growth. 
First,  the  estimate  takes  a  large  number  of  trials  to  accurately  estimate  system 
reliability.  Second,  the  estimator  requires  a  constant  reliability,  R,  for  each  trial. 
Because  the  reliability  at  each  phase,  Rk,  may  not  be  constant  with  k;  only  the  test 
data  from  the  phase  of  interest  may  be  used  to  estimate  reliability.  Failure  discounting 
is  introduced  into  the  model  in  an  attempt  to  adjust  the  underlying  reliability  for  data 
from  previous  phases,  making  it  compatible  with  the  data  from  the  current  phase.  This 
increases   the   amount   of  data   available   for  estimating   reliability   at   each   phase, 
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improving  the  accuracy  of  the  estimate.  Choosing  the  correct  discounting  method  with 
the  correct  parameters  is  critical  to  this  model  because  the  previous  test  data  must  be 
properly  adjusted  to  reflect  an  underlying  reliability  that  is  compatible  with  the  current 
phase.  This  problem  is  difficult  to  solve  because  the  actual  reliability  in  each  phase  is 
unknown. 

Introducing  failure  discounting  creates  a  computation  problem  because  the 
number  of  failures  is  no  longer  an  integer  value.  This  problem  can  be  resolved  in  one 
of  two  ways.  The  failure  can  remain  a  fractional  value  or  the  failure  value  can  be  reset 
to  unity  by  adjusting  the  observed  number  of  trials  up  to  and  including  failure.  Which 
ever  method  is  chosen,  the  ratio  of  failures  to  total  trials  must  remain  constant  or  the 
estimate  of  reliability  will  be  altered.  Returning  the  failure  value  to  unity  and  adjusting 
the  number  of  trials  is  the  method  chosen  to  resolve  this  problem.  This  solution  allows 
the  data  to  retain  its  original  structure  of  the  number  of  trials  up  to  and  including 
failure. 

Adjusting  fractional  failures  and  the  number  of  trials  up  to  and  including 
failure  is  accomplished  by  dividing  both  values  by  the  fractional  failure  value.  This 
returns  the  number  of  failures  to  unity  and  adjusts  the  number  of  trials  such  that  the 
ratio  between  the  number  of  failures  and  the  number  of  trials  remains  constant.  The 
adjusted  number  of  trials  may  not  be  an  integer  value.  This  does  not  present  a  problem 
for  computing  the  MLE.  In  cases  where  the  total  number  of  trials  must  also  be  an 
integer  value,  the  adjusted  number  of  trials  is  rounded  to  the  nearest  integer.  This  will 
have  a  minimal  effect  on  the  ratio  of  failures  to  trials. 

For  example  the  following  data  is  the  result  of  applying  a  failure  discounting 
method  to  observed  test  data. 

FRACTIONAL  FAILURE  =  .5 
NUMBER  OF  TRIALS  =  6 
Dividing  both  values  by  .5  adjusts  the  fractional  failure  to  unity  and  yields  an  adjusted 
failure  value. 

FAILURE  VALUE  =  1.0 
ADJUSTED  NUMBER  OF  TRIALS  =  12 
The  ratio  of  failures  to  trials  remains  constant. 
.5/6  =  1/12  =  .0833 
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In  general,  Equation  2.7  is  used  to  adjust  the  number  of  trials  when  the  adjusted  failure 


OBSERVED  #  of  TRIALS 

ADJUSTED  #  TRIALS  =  (eqn  2.7) 

ADJUSTED  FAILURE 


value  is  returned  to  unity. 

The  reliability  growth  model  is  applied  to  the  data  sequentially,  phase  by 
phase.  After  the  test  data  from  a  phase  is  recorded,  the  discounting  routine  is  applied 
to  the  data  and  an  adjusted  failure  value  is  computed  for  each  observed  failure.  The 
adjusted  number  of  trials  for  each  observed  failure  is  computed  using  Equation  2.7. 


(£  ADJUSTED  TRIALS)  -  J 
X  ADJUSTED  TRIALS 


fik  =  ^  ^TT.__^ „ (eqn  2.8) 


J  -  TOTAL  NUMBER  OF  FAILURES  IN  THE  TEST  PROGRAM  TO  DATE 

Finally,  the  estimate  of  system  reliability  in  phase  k  is  computed  using  Equation  2.8. 
This  procedure  is  repeated  after  each  testing  phase  is  completed  yielding  an  estimate  of 
system  reliability  for  each  phase.  These  estimates  can  then  be  used  to  plot  the 
estimated  reliability  growth  pattern. 

To  illustrate  the  use  of  the  model,  the  discounted  failure  values  from  Table  2 
are  used  to  compute  an  estimate  of  system  reliability.  Because  each  data  point 
represents  a  development  phase,  the  estimate  will  be  of  phase  seven  reliability.  The 
results  of  the  computations  are  shown  in  Table  4. 

Note,  the  sum  of  the  adjusted  trials  is  not  rounded  to  an  integer  value.  Using 
only  the  data  in  the  seventh  phase,  the  MLE  without  discounting  is 

£7  =  7/8  =  .875  . 
The  MLE  using  all  test  data  without  discounting  will  yield 

R7  =   11/18  =  .611  . 
For  this  example,  the  MLE  with  discounting  provides  an  improved  estimate  of  actual 
reliability;  i.e.  Rj  =  .744  is  closer  to  the  actual  reliability  value,  .78. 

When  testing  is  terminated  at  a  fixed  number  of  failures,  the  MLE  has  an 
additional  shortcoming;  it  is  conservatively  biased.  Breaking  the  data  into  sets  of  trials 
up  to  and  including  a  single  failure,  the  number  of  successful  trials  in  a  set  always 
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TABLE  4 

MLE  WITH  DISCOUNTING 

OF  SAMPLE  TEST  DATA  (TABLE  2) 

PHASE  # 

CAUSE 

ADJ.  FAILURE 

ADJ.  TRIALS 

1 

A 

1.0000 

2.0000 

2 

B 

.5625 

1.7778 

1 

j 

C 

.4219 

2.3702 

4 

D 

.3164 

3.1606 

5 

C 

.4219 

4.7405 

6 

B 

.5625 

c    ^  "*  -^  ~% 

7 

A 

1.0000 

8.0000 

k  =  7 

1  = 

27.3824 

J  =  7 

R7  =  (  27.38 
=  .74436 

24  -  7  )  * 

27.3824 

R  =  .78 

equals   the  number  of  trials  in  a  set  minus  one.   As  such,  the  MLE,  Equation  2.2,  may 
be  stated  as  follows  for  each  set  of  data. 


X-l 

R  = 


X 
X  =  NUMBER  OF  TRIALS  UP  TO  AND  INCLUDING  FAILURE 

The  expected  value  of  R  is  derived  in  Equation  2.9.  Table  5  shows  the  E[R]  compared 
to  the  actual  reliability  R.  Clearly,  the  maximum  likelihood  estimate  will  be  very 
conservatively  biased  for  low  reliabilities.   [Ref.  3:  p. 34] 
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E[*]  =  E[(X-1)/X] 
=  1  -  E[  1/X  ] 

E[l/X]  =  (1-R)  +  (R/2)(l-R)  +  (R2/3)(l-R)  +  ... 

=  1  -  R  +  (R/2)  -  (R2/2)  +  (R2/3)  -  (R3/3)  + 

ln(l-R)  =   -R  ~(R2/2)  ~(R3/3)  -... 
(1/R)ln(l-R)  =  1  +  R/2  +  R2/3  +  ... 

.'.E[  1/X]  =  ln(l  -  R)  -  (1/R)ln(l  -  R) 

E[R]  =  1  +  (1/R)ln(l  -  R)  -  ln(l  -  R)        (eqn  2.9) 


TABLE  5 

MLE  EXPECTED  VALUE 

R 

E[fi] 

.1 

.052 

.3 

.168 

.5 

.307 

.7 

.484 

.8 

.598 

.9 

.744 

.99 

.953 

.999 

.993 

This  bias  can  be  offset  by  the  proper  choice  of  a  discounting  parameter;  but, 
the  development  of  a  less  biased  estimator  would  be  advantageous.  The  Exponential 
Single  Phase  Estimate,  introduced  in  the  next  section,  demonstrates  a  smaller 
conservative  bias  than  the  MLE  when  applied  to  geometric  data. 


i 
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2.  Exponential  Single  Phase  Reliability  Estimate 

The  Exponential  Single  Phase  Reliability  Estimate  was  developed  for  the 
special  case  of  testing  until  the  first  failure  is  encountered.  Test  data  of  this  form  has  a 
geometric  distribution  with  the  parameter,  R,  system  reliability.  As  shown  in  the 
previous  section,  the  maximum  likelihood  estimate  is  conservatively  biased  when 
applied  to  geometric  data.  In  an  attempt  to  reduce  this  bias,  an  estimator  of  the 
following  form  was  suggested. 

R  =  1  -  e'A 
e"A  =  1  -  R 
A  =    -ln(l  -  R) 

To  complete  the  development  of  the  estimator,  an  unbiased  estimate,  £, 
for  A  was  sought  such  that 

E[AJ  =   -ln(l  -  R)  =  A 
The  estimate  is  derived  as  follows.   Let  X.  equal  the  number  of  trials  up  to  and 
including  failure  for  the  i th  set  of  geometric  data  and  R.  equal  the  system  reliability  for 
the  i   Ln  set  of  data  where  i   =    1,   2,   3,   ...   ,  number  of  failures.    Assume  that 
Yj  =  f  (  Xj  )  is  an  unbiased  estimate  of  A.  Then 

CO 

Ef(xj)  RjXi~  l  (1-^)=  -ln(l  "  Rj) 

xv=  1 

=  R.  +  R.2  +  R.3  +  ...  . 
i  i  i 

Solving  for  f(  x.  )  by  equating  coefficients  of  terms  with  like  powers  yields 

R  x.  )  =  1  +  1/2  +  1/3  +  ...  +  l/(X.  -  1). 
Resulting  in  Equation  2.10. 

Y.  -  1  +  1/2  +  ...  +1/(X.  -  1)  forXj  >  2         (eqnllO) 

=  0  for  X.  =   1 

i 

Substituting  Y.,  an  unbiased  estimate  of  A,  into  the  reliability  equation  yields  an 
estimate  of  single  phase  reliability,  Equation  2.11.  Equation  2.11  can  be  used  to 
estimate  system  reliability  when  testing  is  performed  until  first  failure. 

As  has  been  shown,  Y.  is  an  unbiased  estimate  of  A;  but,  this  does  not  ensure 
Equation  2.11  will  provide  an  unbiased  estimate  of  system  reliability.  To  resolve  this 
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=  i     •  e-Yi 


R  =  1 


(eqn2.il) 


question    Monte-Carlo    simulation    was    used    to    evaluate    the    properties    of   the 
Exponential  Single  Phase  Estimate  and  compare  it  with  the  MLE. 

The  algorithm  described  below  was  used  to  evaluate  the  Exponential  Single 
Phase  Estimate,  Equation  2.11.  First  a  uniform  (0,1)  random  number  was  generated.  A 
table  look-up  algorithm  was  used  to  convert  the  uniform  (0,1)  random  variable  to  a 
geometric  random  variable  with  reliability  R.  Estimates  of  system  reliability,  based  on 
the  geometric  random  variable,  were  computed  using  both  the  Exponential  Single 
Phase  Estimate  and  the  Maximum  Likelihood  Estimate.  The  procedure  was  then 
replicated  a  large  number  of  times.  The  running  average  of  the  estimates  for  each 
estimator  was  updated  after  each  replication.  The  results  of  the  simulation  are  shown 
in  Table  6  for  ten-thousand  replications. 


TABLE  6 

ESTIMATE  EXPECTED  VALUE 
FOR  MLE  &  EXPONENTIAL  ESTIMATE 

THEORETICAL  RESULTS 

SIMULATION  RESULTS 

R 

E[MLE] 

MEAN  MLE 

MEAN  EXP.  EST. 

.1 

.052 

.053 

.066 

.2 

.107 

.107 

.132 

.3 

.168 

.170 

.207 

.4 

.234 

.234 

.281 

.5 

.307 

.305 

.361 

.6 

.389 

.388 

.452 

.7 

.484 

.484 

.552 

.8 

.598 

.598 

.664 

.9 

.744 

.744 

.798 

.99 

.953 

.954 

.968 

.999 

.993 

.994 

.996 
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The  simulation  generated  mean  for  the  MLE  is  very  close  to  its  theoretically 
derived  value  verifying  the  accuracy  of  the  simulation.  Comparing  the  simulation 
generated  mean  estimate  for  the  Exponential  Single  Phase  Estimator  to  the  actual 
reliability  leads  to  the  conclusion  that  the  estimate  is  conservatively  biased,  but  less 
biased  then  the  MLE.  As  is  the  case  with  the  MLE,  the  amount  of  bias  in  the 
Exponential  Single  Phase  Estimate  is  a  function  of  the  actual  reliability. 

Constant  system  reliability  is  necessary  for  the  Exponential  Single  Phase 
Estimate  to  accurately  predict  reliability.  Because  of  this  property,  the  estimator  must 
be  modified  to  accurately  predict  the  changing  system  reliability  inherent  in  a  reliability 
growth  pattern.  As  was  the  case  with  the  MLE,  the  modification  should  make  it 
possible  to  use  all  available  test  data  when  computing  a  reliability  estimate.  The 
remaining  reliability  growth  models  explore  alternative  modifications  to  the 
Exponential  Single  Phase  Estimate.  The  first  modification  employs  linear  regression  to 
estimate  the  parameter  A.  The  second  modification  involves  taking  the  weighted 
average  of  the  current  single  phase  estimate  and  the  preceding  phase  reliability  estimate 
to  produce  an  estimate  of  current  phase  reliability. 

3.  Exponential  Regression  Discrete  Reliability  Growth  Model 

The  Exponential  Regression  Reliability  Growth  Model  was  developed  by  W. 
M.  Woods  from  the  Exponential  Single  Phase  Reliability  Model  [Ref.  4],  The  model 
studied  here  is  identical  to  the  original  model  with  the  exception  of  applying  failure 
discounting.  The  model  can  only  be  applied  in  situations  where  testing  is  performed  to 
a  fixed  number  of  failures  and  must  have  attribute  data.  Any  failure  discounting 
method  may  be  used  with  the  model;  but,  the  discounted  failure  data  must  be  returned 
to  its  original  form,  an  integer  number  of  failures  and  successes.  To  apply  the  model, 
one  must  record  the  data  necessary  to  apply  the  chosen  failure  discounting  method,  the 
total  number  of  failures  and  successes,  and  the  number  of  successful  trials  between 
each  failure.   The  equation 

R  =  1  -  e"A 
where 

A  =  a  +  pk 
and  k  denotes  the  testing  phase  is  used  to  compute  svstem  reliability 

R  =  1  -  e-(«  +  P  k) 

k  =  1,2,3,... 
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Estimating  the  expression  a  +  Pk  at  each  phase  allows  the  model  to  track,  changing 
reliability.  The  model  is  further  modified  to  include  the  possibility  of  more  than  one 
failure  in  a  given  phase.  Let  Fk  equal  the  total  number  of  failures  in  the  k  th  phase  and 
j  equals  the  failure  number  in  phase  k  such  that  j  =  1,  2,  ...,  Fk.  N:^  equals  the 
number  of  trials  between  the  (j  —  1)  st  and  the  j  l"  failure,  including  the  j  "*  failure,  in 
phase  k.  As  in  the  Exponential  Single  Phase  Estimate,  Equation  2.10,  still  represents 
an  unbiased  estimate  of  (  a  +  (5  k)  for  each  set  of  trials  N:j..  To  avoid  confusion  with 
the  additional  subscript,  Equation  2.10  becomes  Equation  2.12 

Yjk  -  1  +  1/2  +  ...  -H/(Xj  -  1)  forX  >  2         (eqn  2.12) 

=  0  forX.  =  1 

i 

Estimates  of  a  and  P  for  each  phase  are  developed  using  the  techniques  of  linear 

k 
S    (j  -  K.  )  x  Y 

i-i 

Pk     -  (eqn  2.13) 

k 
I    (j  -  K.  )2 

regression  yielding  Equations  2.13  and  2.16.  Equation  2.13  requires  the  computation 
of  Y..  ,  Equation  2.14  ,  and  K.  ,  Equation  2.15. 

Y-k  -  (Ylk  +  Y2k  +  -  +  YFk  k>/Fk  ^n  2'14) 


K.  =  (1  +  2  +  ...  +  k)/k  (eqn  2.15) 


a    =  Y..  -  p.   K.  (eqn  2.16) 


-k 
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Y..  =  (Y.j  +  Y.2  +  ...  +  Y.k  )/k  (eqn  2.17) 

Substituting  the  estimates  for  ak  and  pk  into  the  reliability  equation  yields  the  estimate 
of  reliability  for  phase  k,  Equation  2.18. 

g    =  1  -  e"<ak  +  h^  k  >   1  (eqn  2.18) 


-k 


=  1  -  e'Y-i  k  =  1 


A  separate  definition  of  the  case  where  k=  1  is  necessary  because  regression  techniques 
require  a  minimum  of  two  data  points.   [Ref.  3:  pp.  1-4] 

The  only  modification  made  to  the  original  model  is  the  addition  of  failure 
discounting.  After  each  phase,  the  chosen  failure  discounting  method  is  applied  to  the 
test  data.  The  fractional  failures  that  result  must  be  adjusted  to  unity  by  adjusting  the 
number  of  trials.  Equation  2.7.  Computation  of  Y:^  requires  the  total  number  of  trials 
be  returned  to  an  integer  value.  Therefore,  the  adjusted  trials  are  rounded  to  the 
nearest  integer.  Estimates  for  P,  Equation  2.13,  and  a,  Equation  2.16,  are  then 
computed.  These  values  are  substituted  into  Equation  2.18  to  obtain  the  estimate  of 
system  reliability  for  phase  k. 

Because  the  parameters  a  and  P  are  computed  using  linear  regression,  the 
model  has  the  ability  to  track  changing  reliability  even  if  no  failure  discounting  is 
applied.  Even  so,  it  may  take  numerous  phases  and  a  great  deal  of  data  for  the 
Exponential  Regression  Model  to  accurately  predict  reliability.  Adding  failure 
discounting  to  the  model  should  decrease  the  amount  of  data  required  for  an  accurate 
estimate  of  system  reliability. 

The  Exponential  Regression  Model  can  predict  a  negative  system  reliability. 
This  occurs  most  often  in  the  initial  testing  phases  and  at  low  reliabilities.  Should  this 
occur,  the  estimate  should  be  set  to  zero. 

An  example  will  help  clarify  the  application  of  the  model.  Using  the 
discounted  test  data  from  Table  2,  the  estimate  of  phase  seven  reliability  is  computed 
in  Table  7. 

Note  the  Y-^  column  was  deleted  because  each  phase  consists  of  a  single 
failure  and  Y:^  =  Y.k.  This  example  also  demonstrates  the  results  of  applying 
improper  discounting  parameters  for  the  situation.  Failure  discounting  caused  the 
estimator  to  further  overestimate  the  actual  reliability.  Had  no  discounting  been 
applied,  the  regression  estimate  would  have  yielded  J?7  =  .836. 

31 


TABLE  7 

EXPONENTIAL  REGRESSION  ESTIMATE  COMPUTATIONS 

k      ADJ. 

TRIALS 

Y-k 

(k 

-K.)Y.k 

(k-K.)2 

1 

2 

1.0 

■3.0 

9 

2 

2 

1.0 

■2.0 

4 

3 

2 

1.0 

■1.0 

1 

4 

3 

1.5 

0.0 

0 

5 

5 

2.083 

2.083 

1 

6 

5 

2.083 

4.166 

4 

7 

8 

2.593 

7.779 

9 

s 

11.259 

8.023 

28 

K.  =  4.0 

Y..  =  11.259/7  =  1.608 

P7  =  (8.0 

28  *  28)  =  .287 

a7  =  1.608  -  (.287  x 

4)  =  .460 

(a7  +  P7 

K)  =  .460  + 

(.287  x  7) 

= 

2.469 

fiT-l- 

exp(  -2.469 

)  =  .915 

R7  =  .78 

The    last    remaining    reliability    growth    model    is    also    derived    from    the 
Exponential  Single  Phase  Estimate.  A  weighted  average  technique  is  used  to  allow  the 
estimator  to  track  changing  reliability.  This  estimator  is  described  in  the  next  section. 
4.  Weighted  Average  Discrete  Reliability  Growth  Model 

The  Weighted  Average  Model  tracks  changing  reliability  by  taking  a  weighted 
average  of  the  two  most  recent  reliability  estimates.  This  model  is  only  applicable  to 
situations  when  testing  is  performed  until  a  fixed  number  of  failures  have  been 
observed  and  attribute  data  is  available.  A  slightly  modified  version  of  the  Exponential 
Single  Phase  Reliability  Estimate  is  used  in  the  model.  The  current  phase  reliability  is 
estimated  using  a  weighted  average  of  the  previous  phase  reliability  estimate  and  the 
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current  single  phase  estimate.  The  previous  phase  estimate  and  the  current  single  phase 
estimate  are  weighted  by  the  fraction  of  total  trials  to  date  used  in  the  computation  of 
each  estimate.  As  with  the  Exponential  Regression  Estimate,  this  model  will  track 
changing  reliability  without  the  use  of  failure  discounting;  but,  like  most  weighted 
averages,  the  model  may  lag  actual  growth  by  one  or  more  phases.  Failure  discounting 
is  used  to  correct  the  lag  and  reduce  the  number  of  trials  necessary  to  compute  an 
accurate  estimate. 

Equation  2.11  is  applicable  when  testing  to  first  failure.  A  slightly  different 
version  of  the  estimator  is  applicable  to  situations  where  more  than  one  failure  has 
been  recorded.  To  apply  this  estimator,  a  running  total  of  the  number  of  successful 
trials  to  date,  Sk  ,  and  failures  to  date,  Fk,  is  maintained  for  each  phase  k.  These  totals 
are  used  in  Equation  2.19  to  compute  the  current  single  phase  reliability  estimate, 

-sPk- 

£„n    =  1  -  e-^k         (eqn2.19) 

£k  =   1/Fk  +   ^k  +   !>  +  -  +  lMFk  +  V  "   l  ^ 

For  this  model,  running  totals  of  failure  and  success  data  are  recorded  for 
each  observed  failure  cause.  As  each  trial  is  performed,  the  number  of  successes  for 
each  failure  cause,  except  the  current  failure  cause,  is  incremented  by  one.  Should  a 
previously  recorded  failure  cause  reoccur,  its  failure  total  is  incremented  by  one.  The 
system  values  for  Fk  and  Sk  are  then  selected  from  the  individual  failure  cause  having 
the  largest  number  of  failures  to  date. 

Once  the  single  phase  estimate  is  computed;  the  estimate  of  system  reliability 
is  computed  by  taking  the  weighted  average  of  the  previous  phase  reliability  estimate 

>spk  k=2,3,"  (eqn2.20) 

k=l 

and  the  current  single  phase  estimate,  Equation  2.20. 

Where  N  equals  the  total  number  of  trials  recorded  to  date.  N,  equals  the  total 
number  of  trials  recorded  in  previous  phases.  N2  equals  the  number  of  trials  recorded 
in  phase  k.  Note  the  sum  of  N,  and  N2  equals  N. 
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N1                   N2 

k            N        BjL-1            N 

-E 

=  ~sPk 

Failure  discounting  is  applied  to  the  model  in  much  the  same  way  as  it  was  in 
the  Exponential  Regression  Model.  After  a  discounting  method  has  been  applied  to 
the  data,  fractional  failures  are  returned  to  unity  by  adjusting  the  number  of  trials, 
Equation  2.7.  The  adjusted  number  of  trials  is  then  rounded  to  the  nearest  integer.  It  is 
important  to  note,  when  failure  discounting  is  applied  to  this  model  Sk  must  be 
recomputed  from  phase  one  at  each  phase  boundary  because  the  recorded  number  of 
successful  trials  for  each  previous  phase  may  be  altered  by  the  discounting  routine. 

An  example  of  the  model's  application  should  help  clarify  the  required 
computations.  Using  the  discounted  failure  data  from  the  example  of  the  Straight 
Percent  Discounting  Method  leading  up  to  Table  2,  at  the  end  of  phase  one  the  sum  of 
failures  and  successes  for  the  failure  cause  is  listed  below. 

FAILURES  SUCCESSES 

CAUSE  A  1  1 

Solving  for  the  single  phase  estimate  and  the  reliability  estimate  yield 

Cx  =  1.0 

Rx  =  *spi  =  1  "  e'1  =  .6321  . 
After  the  second  phase  data  is  included  and  the  failure  discounting  method  applied,  the 
sum  of  failures  and  successes  for  each  failure  cause  is  listed  below. 


FAILURES 

SUCCESSES 

CAUSE  A 

1 

2 

CAUSE  B 

1 

2 

From  the  failure  data,  F2  equals  one  and  S2  equals  two  yielding  Q2  equal  to   1.5. 

Applying  the  remaining  model  yields  the  second  phase  reliability  estimate  /?.,. 

gcn    =  1  -  e^3/2)  =  .7^69 
$p2 


Nl  = 

=  2 

N2  = 

=  1 

N  - 

=  3 

*2" 

=     (2/3)  *!     +    (1; 

'3)*sp2 

=  (2/3)(.6321)  + 

(l/3)(.7769) 

=  .6804 
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The  computations  are  repeated  for  each  phase  of  the  testing  program.  As 
mentioned  previously,  when  failure  discounting  alters  the  number  of  successful  trials, 
S. ,  in  a  previous  phase,  the  reliability  estimates  for  all  previous  phases  must  be 
recomputed  to  solve  for  the  current  phase  reliability  estimate.  As  this  simple  example 
clearly  shows,  this  reliability  growth  model  is  computationally  intense.  Programming 
the  model  on  a  computer  may  be  necessary  to  compute  reliability  estimates  for  large 
data  sets. 

Two  failure  discounting  methods  and  three  discrete  reliability  growth  models 
have  been  described  in  this  chapter.  Evaluation  of  each  model  is  very  difficult  because 
the  expected  value  and  variance  of  the  estimates,  when  discounting  is  applied,  become 
mathematically  untractable.  It  is  also  not  possible  to  evaluate  the  models  using  actual 
test  data  because  the  true  reliability  at  each  phase  is  unknown.  The  following  chapter 
discusses  a  Monte-Carlo  simulation  written  to  evaluate  the  performance  of  each  of  the 
failure  discounting  methods  and  discrete  reliability  growth  models. 
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III.  DISCRETE  RELIABILITY  GROWTH  SIMULATION 

Evaluating  reliability  growth  estimators  including  reliability  growth  models,  is 
often  difficult.  True  system  reliability  is  unknown,  rendering  actual  test  data  useless  for 
evaluating  an  estimators  performance.  When  the  mathematics  is  tractable,  the 
properties  of  the  estimator  can  be  derived  and  used  to  evaluate  the  estimators 
performance.  However,  many  estimators  are  too  complex  to  allow  mathematical 
solutions  for  the  closed  form  of  their  properties.  For  this  group  of  estimators, 
computer  simulation  is  one  of  the  only  evaluation  techniques  available.  The  addition  of 
failure  discounting  to  an  estimator  makes  mathematical  solutions  very  difficult  if  not 
impossible.  The  reliability  growth  models  introduced  in  this  study  fall  in  the  category 
requiring  simulation  as  an  evaluation  tool. 

A  Monte-Carlo  simulation  was  written  to  evaluate  the  properties  of  the  three 
reliability  growth  models  introduced  in  Chapter  II.  The  simulation  builds  a  reliability 
growth  pattern  and  generates  test  data  from  the  known  system  reliabilies  using  the 
geometric  distribution.  Each  of  the  reliability  growth  models  are  then  applied  to  the 
simulation  generated  test  data.  Repeating  the  process  many  times  for  a  particular  set  of 
known  reliabilities  will  provide  a  good  estimate  of  both  the  mean  and  variance  of  each 
estimator  as  well  as  provide  information  about  the  underlying  distribution  of  the 
estimate.  These  statistics  can  then  be  used  to  evaluate  each  reliability  growth  model's 
performance. 

Several  modeling  problems  had  to  be  solved  before  the  simulation  could  be  built. 
A  method  for  generating  a  realistic  reliability  growth  pattern  had  to  be  developed.  This 
method  must  closely  model  the  process  of  testing  equipment,  implementing  design 
improvements  or  "fixes"  and  continued  testing.  It  also  must  providing  the  flexibility  to 
test  numerous  reliability  growth  patterns.  Once  the  actual  system  reliability  in  each 
phase  is  established  from  the  growth  pattern,  test  data  of  the  form  the  number  of  trials 
up  to  and  including  failure  must  be  generated.  Because  such  data  are  geometrically 
distributed  when  R  is  fixed,  this  process  is  relatively  simple.  A  relistic  method  must  also 
be  developed  for  determining  the  cause  of  each  failure.  This  step  proved  to  be  the 
most  difficult. 
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As  with  all  simulations,  several  modeling  assumptions  were  necessary.  The 
following  section  discusses  the  assumptions  and  their  impact  on  the  simulation.  The 
remainder  of  the  chapter  describes  the  development  of  the  simulation  algorithm  and  its 
capabilities  and  limitations. 

A.       MODELING  ASSUMPTIONS 

Building  an  algorithm  that  will  generate  realistic  test  data  requires  several 
simplifying  assumptions  about  the  actual  test  process.  These  assumptions  can  be 
broken  into  three  categories.  They  are: 

1.  General  assumptions 

2.  Assumptions  concerning  the  true  reliability  growth  pattern 

3.  Assumptions  concerning  failure  cause  determination 

The  last  two  categories  correspond  to  the  two  steps  in  generating  test  data,  generating 
the  actual  reliability  growth  pattern  and  computing  test  data  from  the  growth  pattern. 
Each  assumption  is  listed  and  discussed  by  category. 

1.  General  Assumptions 

The  general  assumptions  narrow  the  scope  of  testing  considered  to  those 
scenarios  which  apply  to  the  reliability  growth  models  evaluated.  The  two  required 
assumptions  are: 

1.  All  testing  is  performed  to  a  fixed  number  of  failures. 

2.  All  test  results  are  recorded  as  attribute  data. 

These  assumptions  ensure  the  test  data  produced  by  the  simulation  will  be  compatible 
with  the  discrete  reliability  growth  models  developed  in  the  study.  The  models  can  only 
be  applied  to  discrete  or  attribute  data  where  testing  is  performed  until  a  fixed  number 
of  failures  are  observed. 

2.  Reliability  Growth  Pattern  Assumptions 

Defining  the  mechanisms  which  cause  an  items'  reliability  to  change  from 
phase  to  phase  is  necessary  before  an  algorithm  to  generate  a  realistic  reliability 
growth  pattern  can  be  written.  Several  assumptions  were  necessary  to  properly  define 
these  mechanisms.  They  are: 

1.  The  reliability  growth  pattern  is  non-decreasing. 

2.  System  reliability  changes  only  at  phase  boundaries. 

3.  Equipment  improvements  are  implemented  immediately  after  a  phase  ends  and 
before  any  further  testing. 
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4.  Only  design  weaknesses  causing  failure  during  a  phase  are  corrected  at  the  end 
of  a  phase. 

5.  Each   design  improvement   removes   a  fixed   fraction   of  the   probability   of 
reoccurrence  for  the  corresponding  failure  cause. 

The  first  assumption  limits  the  shape  of  the  generated  growth  patterns  to  non- 
decreasing  curves.  Because  one  of  the  objectives  of  development  testing  is  to  improve 
system  reliability,  this  assumption  is  logical.  However,  actual  reliability  growth  may 
have  periods  of  decline  when  "fixes"  to  design  problems  actually  decrease  reliability. 
Over  the  entire  testing  period,  even  these  curves  are  generally  increasing.  The  worst 
case  this  assumption  allows  is  no  growth  over  the  testing  phases.  This  should  be 
adequate  for  determining  adverse  growth  characteristics  of  the  reliability  growth 
models. 

Assuming  system  reliability  remains  constant  throughout  a  testing  phase 
makes  the  model  consistent  with  the  previous  definition  of  a  testing  phase.  This 
assumption  means  no  design  changes  or  any  other  improvements  are  made  to  an  item 
on  test  within  a  phase.  Because  a  phase  can  be  defined  as  any  number  of  failures 
greater  than  or  equal  to  one,  this  assumption  should  have  little  impact  upon  the 
accuracy  of  the  simulation. 

Assuming  all  fixes  are  implemented  after  a  test  phase  ends  and  before  the  next 
phase  begins  is  closely  related  to  the  previous  assumption.  This  assumption  does 
preclude  accurate  modeling  of  long  term  fixes  whose  implementation  are  postponed 
while  testing  continues.  The  impact  of  this  limitation  should  be  minimal. 

Assumption  four  asserts  that  the  only  known  weaknesses  in  system  design  are 
those  which  have  caused  system  failure.  This  is  not  always  the  case.  Suspected 
weaknesses  may  be  corrected  along  with  those  weaknesses  confirmed  by  system  failure. 
The  simulation  is  unable  to  model  this  situation.  Even  though  this  deviates  somewhat 
from  reality,  most  major  improvements  to  system  reliability  should  come  from  failure 
identified  weaknesses.  The  intent  of  the  simulation  is  only  to  generate  a  realistic 
reliability  growth  pattern  not  to  duplicate  an  actual  growth  pattern.  Therefore  the 
simulation  results  should  be  adequate  to  accomplish  the  task  for  which  the  model  was 
designed. 

The  last  assumption  in  this  category  deals  with  the  mechanism  used  by  the 
simulation  to  model  improvements  in  system  design.  It  asserts  that  all  fixes  remove  a 
constant  fraction  of  the  probability  of  occurrence  for  the  failure  cause  of  interest.  In 
reality  many  factors  determine  how  effective  a  particular  fix  will  be  and  all  fixes  are  not 
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equally  effective.  This  method  also  decreases  the  probability  of  occurrence  more  for  a 
failure  cause  with  a  high  probability  of  occurrence  than  for  a  failure  cause  with  a  low 
probability  of  occurrence.  This  also  is  unrealistic.  Despite  the  shortcomings  of  this 
simplification,  it  does  provide  a  simple  way  to  model  reliability  improvement  due  to 
design  change  and  will  produce  a  growth  pattern  that  is  adequate  for  evaluating  the 
reliability  growth  models  studied. 

The  final  category  of  assumptions  deal  with  generating  test  data  from  a  known 
system  reliability  and  determining  the  cause  of  each  failure.  They  are  discussed  in  the 
next  section. 

3.  Failure  Cause  Determination  Assumptions 

Realistically  modeling  failure  causes  is  critical  to  the  evaluation  of  the  failure 
discounting  methods.  The  technique  used  to  model  failure  cause  determination  must 
also  be  incorporated  into  the  technique  used  to  model  reliability  growth  because  the 
two  processes  are  closely  related.  The  assumptions  made  in  modeling  failure  cause 
determination  are: 

1.  There  exists  a  finite  number  of  possible  failure  causes. 

2.  Each  failure  cause  has  a  fixed  probability  of  occurrence  in  each  phase. 

3.  System  reliability  can  be  modeled  as  a  series  system  of  the  failure  causes. 

4.  Each  failure  cause  is  stochastically  independent  of  the  other  failure  causes. 

Assuming  a  fixed  number  of  failure  causes  may  be  an  oversimplification  of 
reality  but  should  not  invalidate  the  model.  Even  though  the  possible  number  of  failure 
causes  is  most  likely  infinite,  only  a  finite  number  of  these  causes  with  relatively  high 
probabilities  of  occurrence  are  of  interest.  The  number  of  possible  failure  causes 
considered  by  the  simulation  must  be  specified  as  an  input  to  the  model;  but,  the 
number  of  causes  input  is  only  limited  by  the  capacity  of  the  computer  on  which  the 
model  is  run.  Therefore,  the  simulation  is  flexible  enough  to  model  nearly  any  scenario 
of  interest. 

Fixing  the  probability  of  occurrence  for  all  failures  causes  within  a  single 
phase  is  a  direct  result  of  fixing  the  system  reliability  for  all  trials  in  a  single  phase. 
This  assumption  will  not  further  restrict  the  ability  of  the  algorithm  to  model  reality. 

An  assumption  is  needed  to  interrelate  failure  cause  determination  and  system 
reliability  growth  through  design  improvements.  Assuming  the  component  being  tested 
may  be  modeled  as  a  series  system  of  failure  causes  meets  this  requirement.  System 
reliability  at  any  phase  is  a  function  of  the  probability  of  occurrence  for  the  specified 
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failure  causes.  As  such,  implementing  a  design  improvement  reduces  the  probability  of 
occurrence  for  a  failure  cause  which  in  turn  increases  the  systems  reliability.  This 
assumption  also  makes  it  possible  to  stochastically  determine  the  cause  of  failure.  The 
mathematical  derivation  of  this  technique  "will  be  presented  in  a  later  section. 
Modeling  the  failure  cause  as  a  series  system  does  not  depart  far  from  reality  because 
the  occurrence  of  a  failure  cause  implies  system  failure.  Once  the  assumption  of  a 
finite  set  of  failure  causes  is  accepted,  modeling  them  as  a  series  system  does  not 
further  restrict  the  models  representation  of  reality. 

The  final  assumption  requires  that  each  failure  cause  be  stochastically 
independent.  This  means  that  the  causes  of  failure  are  not  related.  The  occurrence  of 
one  cause  does  does  not  influence  the  probability  of  occurrence  for  any  other  cause.  If 
the  failure  causes  are  properly  defined,  this  assumption  should  not  limit  the  accuracy  of 
the  model. 

The  following  section  discusses  the  simulation  algorithm  built  from  these 
assumptions. 

B.       SIMULATION  ALGORITHM  DEVELOPMENT 

The  algorithm  developed  for  this  simulation  can  be  broken  into  the  four  sections. 
They  are: 

1.  Generate  a  reliability  growth  pattern. 

2.  Generate  test  data,  including  the  cause  of  failure,  from  the  reliability  growth 
pattern. 

3.  Apply  the  failure  discounting  methods  and  reliability  growth  models  to  the 
simulation  generated  data. 

4.  Compute  statistics  used  to  evaluate  the  performance  of  each  model. 

The  first  two  steps  of  the  algorithm  along  with  the  output  statistics  will  be 
discussed  in  detail  in  this  section.  The  methodology  used  to  apply  each  of  the 
discounting  methods  and  reliability  growth  models  was  discussed  in  Chapter  II  and  will 
receive  little  attention  here.  With  few  exceptions,  the  actual  content  of  the  Fortran 
code  used  in  the  simulation  will  not  be  discussed.  A  documented  copy  of  the  code  is 
included  at  Appendix  A  for  the  interested  reader. 

1.  Simulation  of  a  Reliability  Growth  Pattern 

The  reliability  growth  pattern  built   by  the   simulation  models  the   actual 
process  of  testing  and  improving  an  item  in  a  development  program.    Several  user 
inputs  are  required  at  this  stage  of  the  simulation.  They  are: 
1.     number  of  testing  phases 
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2.  number  of  failures  in  each  testing  phase 

3.  number  of  possible  failure  causes 

4.  the  first  phase  probability  of  occurrence  for  each  possible  failure  cause 

5.  growth  fraction  (F  ) 

Once  these  parameters  are  in  place,  the  simulation  computes  the  first  phase 
system  reliability  from  the  probability  of  occurrence  for  each  possible  failure  cause.  The 
number  of  trials  up  to  and  including  failure  and  the  failure  cause  are  then  computed  for 
all  failures  in  the  first  phase.  The  algorithm  used  to  generate  system  reliability  and  the 
failure  data  will  be  covered  in  the  next  section. 

After  the  failure  data  for  the  first  phase  has  been  generated,  the  probability  of 
occurrence  for  each  failure  cause  which  occurred  in  the  first  phase  is  recomputed  using 
the  growth  fraction.  Assume  the  first  failure  occurred  due  to  cause  A,  the  new 
probability  of  occurrence,  Pr  ( occurrence )  ,  will  be  computed  using  Equation  3.1. 

Pr  (occurrence)n  =  Pr( occurrence)  x  (1  —  F)  (eqn3.1) 

This  equation  is  applied  at  the  end  of  each  phase  to  those  failure  causes  that  occurred 
in  the  current  phase.  If  a  failure  cause  is  responsible  for  multiple  failures  in  the  phase, 
Equation  3.1  is  applied  only  one  time.  This  procedure  simulates  identifying  a  system 
weakness  through  testing  and  implementing  a  design  improvement.  The  procedure  is 
repeated  at  the  end  of  each  testing  phase. 

The  result  of  the  algorithm  is  a  stochastically  determined  reliability  growth 
pattern  that  simulates  actual  developmental  testing.  A  test  is  performed,  weaknesses  in 
the  system  are  identified,  corrective  design  changes  are  implemented  and  testing  is 
continued.  This  algorithm  also  has  the  advantage  of  interrelating  the  probabilities  of 
occurrence  for  each  failure  cause  and  the  system  reliability.  This  will  be  useful  in 
determining  the  failure  cause.  By  changing  the  growth  fraction,  the  rate  of  reliability 
growth  can  be  varied  from  no  growth  to  rapid  growth. 
2.  Generating  Test  Data  From  Known  Reliability 

Two  problems  must  be  solved  before  an  algonthm  that  generates  test  data  can 
be  developed.  The  system  reliability  must  be  determined  from  the  probability  of 
occurrence  for  each  failure  cause.  A  technique  for  generating  the  number  of  trials  up  to 
and  including  failure  from  known  reliability  that  identifies  the  failure  cause  also  must 
be  developed.    The  solution  to  both  problems  lies  with  the  assumption  that  system 
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reliability  can  be  modeled  as  a  series  system  of  the  failure  causes.  In  a  discrete  series 
system  the  number  of  trials  up  to  and  including  system  failure  will  be  the  minimum  of 
the  number  of  trials  up  to  and  including  failure  for  each  sub-component.  Let  M  equals 
the  number  of  trials  up  to  and  including  failure  for  the  system  and  X.  where 
i=  1  ,2  ,  ...  ,  j  denotes  the  number  of  trials  up  to  and  including  failure  for  the  i  m 
subcomponent,  here  a  failure  cause.  If  the  j  components  form  a  series  system,  M  may 
be  expressed  as 

M  =  MIN  (XVX2,...,X.J 
Because  the  probability  of  occurrence  is  fixed  throughout  a  phase  for  each  failure 
cause,  X.  is  distributed  as  a  geometric  random  variable  with  parameier  R..  From  an 
assumption,  the  X.'s  are  also  mutually  independent.  It  can  be  shown  for  a  geometric 

random  variable,  X.,  that 

'     1' 

Pr  (Xj  >  n)  =  Rjn. 
The  distribution  for  M  is  derived  as  follows. 
M  =  MIN  [  XVX2,...,X.  ] 
Pr  [M  >  n]  =  Pr  [  X{  >  n,  X2  >  n,  ...  ,X:  >  n  ] 

=  ?v[X{  >  n]  x  pr[X2  >  n]  x  ...  x  pr[X-  >  n] 

=   R,n   X    R  n   x   ...    x    R.n 

1  2  j 

=   [  Rj    X    R2    X    ...    X  -R.  f 

Therefore  M  is  also  distributed  as  a  geometric  random  variable  with  the  parameter 
R  =  (  R}   x  R2  x  ...  x  R.  )  . 

The  system  reliability  at  each  phase  is  computed  as  the  product  of  the 
compliments  of  the  probabilities  of  occurrence  for  each  possible  failure  cause.  Because 
of  this  computation  the  actual  simulation  input  is  the  compliment  of  the  first  phase 
probability  of  occurrence  for  each  failure  cause.  As  the  probability  of  occurrence  for 
each  failure  cause  is  recomputed  from  phase  to  phase,  the  system  reliability  is  also 
adjusted. 

The  number  of  trials  up  to  and  including  failure  for  a  geometrically  distributed 
random  variable  with  reliability  R  can  be  generated  from  a  uniform  (0,1)  random 
variable,  U,  using  Equation  3.2.   [Ref.  5]. 

X  =  INT  [  1  +  (  ln(U)/ln(R)  )  ]  (eqn  3.2) 
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Equation  3.2  combined  with  the  definition  of  a  series  system  provide  the  solution  to 
generating  the  number  of  trials  up  to  and  including  failure  while  accounting  for  the 
cause  of  failure. 

For  each  failure  generated  by  the  simulation,  Equation  3.2  is  used  to  generate 
a  geometric  random  variable  for  each  failure  cause  using  the  compliment  of  the 
probability  of  occurrence  as  the  parameter.  The  minimum  geometric  random  variable 
over  the  failure  causes  becomes  the  number  of  trials  up  to  and  including  failure  for  the 
system.  The  cause  of  system  failure  is  recorded  as  the  failure  cause  producing  the 
minimum  geometric  random  variable.  Generating  the  test  data  in  this  fashion  allows 
the  reliability  growth  pattern  and  test  data  to  be  stochastically  generated  yet 
interrelated  much  as  they  are  in  nature.  A  summary  of  the  steps  required  to  compute 
the  reliability  growth  pattern  and  the  test  data  follows. 

1.  Input  the  required  data 

2.  Generate  phase  failure  data  including  the  number  of  trials  and  the  failure  cause. 

3.  Recompute  the  probability  of  occurrence  for  the  failure  causes  that  occur  in  the 
phase. 

4.  Return  to  step  2  until  all  phases  are  complete. 

Once  the  test  data  has  been  computed  from  the  known  reliability  in  each 
phase,  the  failure  discounting  methods  and  reliability  growth  models  can  be  applied  to 
the  test  data.  The  resulting  reliability  estimates  can  then  be  evaluated  against  the 
known  system  reliability. 

3.  Application  of  the  Discounting  Methods  and  Reliability  Growth  Models 

The  discounting  methods  and  reliability  growth  models  are  applied  to  the 
generated  test  data  in  much  the  same  way  as  they  were  described  in  Chapter  II.  Each 
of  the  three  reliability  growth  models  are  applied  to  each  set  of  test  results.  However, 
the  simulation  user  must  specify  one  of  the  two  failure  discounting  methods  because 
the  simulation  can  not  apply  both  simultaneously.  The  maximum  likelihood  estimate 
without  discounting  is  also  computed  for  each  phase,  independent  of  all  other  test  data. 
This  allows  a  comparison  between  the  standard  reliability  estimator  and  the  reliability 
growth  .models.  The  simulation  produces  an  estimate  of  system  reliability  from  each 
reliability  growth  model  for  each  testing  phase.  The  single  phase  MLE  and  actual 
system  reliability  are  provided  as  a  basis  for  comparison. 

One  minor  departure  from  the  application  of  the  reliability  growth  models  as 
described  in  Chapter  II  was  necessary.  For  large  numbers  of  trials  up  to  and  including 
failure,  N,  computations  of  the  form 
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Y  =  1  +  (1/2)  +  ...  +  [1/(N-1)] 
required  extensive  computer  processing  time  to  solve  recursively.  To  improve  program 
efficiency,  the  values  for  Y  up  to  and  including  one-thousand  were  computed  exactly 
and  stored  in  an  array.  For  values  of  N  greater  than  one-thousand  non-linear 
regression  was  used  to  estimate  the  value  of  Y.  Equation  3.3  provided  an  adequate 
estimate  with  an  R2  value  of  .99635. 


Y  =  3.55445  x  n(U/100) 


(eqn  3.3) 


> 
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Figure  3.1     Regression  Estimate  of  Y. 

Figure  3.1  compares  the  actual  Y  value  with  the  regression  predicted  value  for  large  N. 
Both  the  Weighted  Average  Model  and  the  Exponential  Regression  Model  are  affected 
by  the  approximation.  The  approximation  has  minimal  impact  upon  the  computation 
of  model  estimates.  Values  for  N  above  one-thousand  are  rare  even  at  high  reliability 
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and  values  above  15,000,  the  accurate  extension  of  the  approximation,  are  very  rare. 
Should  a  value  of  N  larger  than  15,000  occur,  some  error  will  be  introduced  into  the 
model  estimate,  and  this  error  will  increase  with  N.  Because  these  values  are  rare,  this 
error  is  not  considered  significant. 

Further  research,  after  the  simulation  was  written,  has  yielded  a  more  accurate 

Y  =  C+loge(N)+l/(2N)-l/[12N(N+l)]-l/[12N(N4-l)(N+2)]         (eqn  3.4) 
C  =  Euler's  Number  =  0.5772156648 

approximation  for  Y.  This  series  solution  is  shown  in  Equation  3.4  [Ref.  6].  This 
equation  should  be  incorporated  into  the  simulation  to  improve  accuracy  in  future 
runs. 

The  estimates  resulting  from  the  application  of  the  reliability  growth  models 
are  retained  through  the  desired  number  of  replications  to  allow  the  computation  of 
descriptive  statistics.  These  statistics  aid  in  evaluating  each  growth  model. 
4.  Statistical  Summary  of  Model  Results 

Several  descriptive  statistics  are  generated  by  the  model  that  aid  in  evaluating 
the  performance  of  each  reliability  growth  model.  They  include  the  mean  and  standard 
deviation  of  the  reliability  estimates  at  each  phase  for  each  of  the  growth  models.  A 
confidence  interval  is  also  computed  for  the  mean  estimate  at  each  phase. 

Replications  of  the  simulation  are  required  to  compute  the  descriptive 
statistics.  After  the  first  replication,  the  actual  system  reliability  in  each  phase  is  fixed. 
At  each  replication,  new  test  data  is  generated  from  the  fixed  reliability  growth  pattern 
and  each  of  the  reliability  growth  models  is  applied  to  the  data.  The  estimates 
generated  by  the  growth  models  are  retained  at  each  replication.  When  the  desired 
number  of  replications  have  been  completed,  the  sample  mean  and  sample  standard 
deviation  for  each  estimator  are  computed  using  standard  formulas.  A  95%  confidence 
interval  for  the  mean  estimate  is  computed  for  each  phase  reliability  estimate  with  the 
assumption  that  each  estimate  is  normally  distributed.  The  assumption  of  normality  is 
not  valid  in  every  case  but  the  confidence  interval  is  included  as  a  tool  for  comparison. 
Further  analysis  of  the  reliability  growth  models  can  be  accomplished  by  saving  each 
estimate  in  an  output  file.  The  models  user  can  then  analyze  the  data  as  desired. 

Changing  the  simulations'  input  parameters  makes  it  possible  to  analyse  a 
wide  variety  of  testing  situations.  These  capabilities  are  discussed  in  the  following 
section. 
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C.       CAPABILITIES  OF  THE  MODEL 

Within  the  limits  of  the  assumptions  stated  earlier,  the  simulation  is  very  flexible. 
Any  number  of  failure  causes  may  be  input  into  the  model.  By  varying  the  phase  one 
probability  of  success  for  each  failure  cause,  the  growth  fraction  and  the  number  of 
phases  a  wide  range  of  reliability  growth  curves  may  be  explored.  The  random  number 
generator  seed  is  also  user  specified  allowing  the  exact  string  of  random  numbers  and 
test  data  to  be  duplicated.  In  this  way,  various  discounting  methods  and  parameters 
may  be  evaluated  using  the  exact  same  data. 

The  test-fix-test  scenario  may  be  modeled  by  specifying  a  single  failure  per  phase. 
The  test-fmd-test  methodology  may  be  simulated  by  specifying  multiple  failures  in  each 
phase.  There  is  no  requirement  that  each  phase  have  an  equal  number  of  failures. 
Therefore  a  mixed  scenario,  test-fix-test  and  test-find- test,  may  be  simulated. 

Each  of  the  discounting  methods  described  in  Chapter  II  may  be  applied  to  the 
data  using  the  full  range  of  input  parameters.  Proper  parameter  selection  for  the 
discounting  method  will  cause  no  discounting  to  occur.  In  this  manner  each  of  the 
reliability  growth  models  may  be  evaluated  without  failure  discounting  applied. 

The  simulation  provides  a  tool  for  evaluating  the  reliability  growth  models 
against  a  known  system  reliability  growth  pattern.  When  large  numbers  of  replications 
are  run,  information  concerning  the  expected  value  and  variance  of  each  estimator  may 
be  obtained.  The  combinations  of  failure  discounting  parameters,  reliability  growth 
models  and  anticipated  growth  patterns  may  also  be  evaluated.  The  following  chapter 
discusses  the  results  of  numerous  runs  of  the  simulation  with  varying  growth  patterns 
and  failure  discounting  parameters. 
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IV.  EVALUATION  OF  RELIABILITY  GROWTH  MODELS 

The  discounting  methods  and  reliability  growth  models  were  evaluated  using  the 
Monte-Carlo  simulation  described  in  Chapter  III.  The  results  from  numerous  runs  of 
the  simulation  with  varying  reliability  growth  patterns,  failure  discounting  methods  and 
failure  discounting  parameters  were  reduced  to  graphical  output  for  evaluation. 

The  purpose  of  the  evaluation  was  to  determine  general  characteristics  of  each 
reliability  growth  model.  The  number  of  simulation  runs  was  not  adequate  to  prove 
that  conclusions  reached  here  will  hold  for  all  cases.  Also,  no  fast  rules  for  model 
selection  or  failure  discounting  parameter  selection  were  developed.  Still,  many 
characteristics  of  the  different  reliability  growth  models  were  apparent  from  the 
simulation  runs  completed.  In  particular  the  answers  to  the  following  questions  were 
sought. 

1.  How  well  does  the  reliability  growth  model  track  the  actual  reliability  growth 
pattern? 

2.  How  did  the  choice  of  failure  discounting  parameters  affect  the  estimates  of 
reliability? 

3.  How  variable  were  the  estimates  provided  by  each  of  the  models? 

It  was  impossible  to  evaluate  all  possible  reliability  growth  patterns  that  can  be 
produced  by  the  simulation.  Five  widely  different  reliability  growth  patterns  were 
chosen  to  be  representative  of  the  possible  range.  Constant  reliability  cases  were 
included  to  identify  models  that  predict  reliability  growth,  were  in  fact,  none  exists.  The 
five  reliability  growth  patterns  chosen  are  described  below. 

1 .  low  reliability  with  no  growth 

2.  high  reliability  with  no  growth 

3.  low  initial  reliability  with  moderate  growth 

4.  moderate  initial  reliability  with  moderate  growth 

5.  low  initial  reliability  with  rapid  growth 

All  simulation  input  parameters  were  kept  fixed  to  insure  the  different  simulation 
runs  could  be  compared.  The  number  of  phases  was  held  constant  at  ten  with  one 
failure  per  phase.  Five  failure  causes  were  used  and  each  simulation  was  replicated  one- 
hundred  times.  The  seed  for  the  random  number  generator  was  kept  constant  for  all 
simulation  runs  insuring  the  same  string  of  random  numbers  and,  therefore,  identical 
test  data  was  generated  for  each  growth  pattern. 
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Each  reliability  growth  model  is  evaluated  separately  in  the  following  sections. 
The  graphical  results  of  the  remaining  simulation  runs  are  included  in  Appendix  B. 

A.  GENERAL  RESULTS 

Characteristics  common  to  all  the  reliability  growth  models  are  stated  here  for 
convenience. 

The  variability  of  each  of  the  estimators  decreased  with  increasing  phases.  This  is 
due  to  the  increased  data  available  to  the  model  with  each  additional  phase. 

The  accuracy  of  each  estimate  increased  and  the  variability  of  each  estimate 
decreased  with  increasing  reliability.  This  is  a  function  of  testing  until  a  fixed  number 
of  failures  have  been  observed.  As  reliability  increases,  the  average  number  of  trials 
required  to  produce  failure  will  also  increase.  Therefore,  as  reliability  increases,  more 
data  is  available  for  the  model  improving  the  accuracy  of  the  estimate  and  decreasing 
the  estimate's  variability. 

B.  WEIGHTED  AVERAGE  RELIABILITY  GROWTH  MODEL 

The  Weighted  Average  Reliability  Growth  Model  proved  to  be  ineffective  as  an 
estimator  of  changing  reliability.  The  model  consistently  overestimated  the  actual 
reliability  even  when  no  failure  discounting  was  applied.  Figure  4.1  plots  the  mean 
estimate  from  each  model  at  each  phase.  Clearly  the  mean  phase  estimates  for  the 
Weighted  Average  Model  overestimated  the  actual  reliability  at  each  phase.  The 
addition  of  failure  discounting  only  amplified  the  tendency  to  overestimate  as  seen  in 
Figure  4.2.  Even  though  the  variance  of  the  estimates  decreased  rapidly  with 
increasing  phases,  the  model  is  not  useful  in  its  current  form.  Therefore,  the  Weighted 
Average  Model  was  not  considered  in  anv  further  analvsis. 
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C.       MAXIMUM  LIKELIHOOD  ESTIMATE  WITH  FAILURE  DISCOUNTING 

The  MLE  With  Discounting  Model  accurately  tracked  a  wide  range  of  reliability 
growth  patterns.  With  proper  selection  of  the  failure  discounting  method  and 
parameter,  this  model  tracked  each  of  the  reliability  growth  patterns  tested.  Figures 
4.2,  4.3,  4.4,  4.5,  and  4.6  display  the  model's  performance  for  each  of  the  five  reliability 
growth  patterns  tested.  The  model  is  very  sensitive  to  the  selection  of  the  failure 
discounting  method  and  parameter.  In  general,  the  model  underestimates  actual 
reliability  in  the  early  phases  and  slowly  converges  to  the  actual  value  with  increased 
test  data. 
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Figure  4.1     Reliability  Growth  Models 
Low  Constant  Reliability 
No  Discounting  Applied. 

The  choice  of  failure  discounting  method  and  parameter  is  critical  to  the  MLE 
With  Discounting  Model.  Because  the  Lloyd  Discounting  Method  does  not  have  a 
discounting  interval,  it  generally  reduces  the  discounted  failure  value  more  rapidly  than 
the  Straight  Percent  Discounting  Method.  This  results  in  rapid  growth  of  the  reliability 
estimates.  The  Standard  Discounting  Method  with  the  discount  interval  set  at  one  and 
the  proper  choice  of  the  discount  fraction  will  provide  nearly  identical  results  to  the 
Lloyd  method.  For  all  the  reliability  growth  patterns  tested,  except  low  initial  reliability 
with  rapid  growth,  the  Lloyd  discounting  Method  caused  the  model  to  overestimate  the 
actual  reliability.  The  overestimation  was  particularly  pronounced  for  low  constant 
reliability  where  the  Lloyd  Discounting  Method  forced  the  model  to  overestimate 
reliability  even  with  y  as  large  as  .99999,  Figure  4.7.  Because  of  this  characteristic,  the 
Lloyd  discounting  Method  is  not  recommended  for  use  with  the  Maximum  Likelihood 
Estimate. 
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Figure  4.2    Reliability  Growth  Models 
Low  Constant  Reliability 
Std.  Discounting  Applied. 

With  the  addition  of  the  second  parameter,  the  discount  interval,  the  Straight 
Percent  Discount  Method  is  more  flexible  than  the  Lloyd  Method.  Even  so,  it  has  the 
potential  to  overestimate  the  system  reliability.  The  choice  of  model  parameters  is 
critical  to  the  model's  performance.  Figure  4.8  demonstrates  the  change  in  model 
estimates  with  varying  discounting  parameters.  Several  factors  influence  the  proper 
choice  of  parameters.  Three  of  the  most  important  are  the  amount  of  testing  to  be 
performed,  estimates  of  system  reliability  before  testing  begins  and  how  rapidly 
reliability  is  expected  to  grow  throughout  the  development  and  testing  program.  No 
rales  have  been  developed  to  help  select  the  proper  discounting  parameters.  Experience 
with  the  model,  good  engineering  judgement  and  possibly  simulation  are  necessary  to 
select  a  combination  of  discounting  parameters  that  will  cause  the  model  to  accurately 
track  the  actual  reliability  growth  pattern.  Even  an  apparently  good  choice  of  failure 
discounting    parameters    may    result    in    erroneous    reliability    estimates    if  the    true 
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Figure  4.3     Reliability  Growth  Models 
High  Constant  Reliability 
Std.  Discounting  Applied. 

reliability  does  not  grow  as  expected.  Therefore,  when  using  this  model,  frequent 
engineering  evaluations  may  be  necessary  to  insure  design  evolution  is  progressing  as 
expected. 

Reliability  estimates  from  the  Maximum  Likelihood  Estimate  With  Discounting 
Model  exhibited  the  smallest  variance  of  the  models  evaluated.  Figures  4.9  and  4.10 
are  box  plots  of  the  one  hundred  reliability  estimates  at  each  testing  phase  for  two  of 
the  reliability  growth  patterns.  Comparison  with  other  models  may  be  made  from  the 
additional  box  plots  included  at  Appendix  B.  The  reliability  estimates  from  all  three 
models  exhibited  large  variance  in  the  first  few  phases  of  testing.  The  variance  of  the 
estimates  gradually  decreased  as  more  test  data  became  available.  With  few  exceptions, 
the  MLE  With  Discounting  exhibited  the  smallest  variability  over  the  entire  range  of 
phases  for  each  reliability  growth  pattern  tested.  This  reduced  variance  was 
particularly  pronounced  in  the  early  phases  and  at  low  reliability. 
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Figure  4.4    Reliability  Growth  Models 

Low  Reliability  Moderate  Growth 

Std.  Discounting  Applied. 

The  MLE  With  Discounting  Model  can  accurately  track  most  reliability  growth 
patterns  when  the  Straight  Percent  Discounting  Method  is  used.  It  does  so  with  equal 
or  less  variance  than  the  other  models  evaluated.  However,  the  model  is  very  sensitive 
to  the  choice  of  of  failure  discounting  parameters  and  may  either  over  or  under 
estimate  the  actual  reliability  if  the  parameters  are  not  chosen  correctly. 

D.       EXPONENTIAL  REGRESSION  MODEL 

The  Exponential  Regression  Model  has  the  ability  to  accurately  track,  most 
reliability  growth  patterns,  Because  it  uses  a  combination  of  linear  regression 
techniques  and  failure  discounting  to  track  changing  reliability,  it  reacts  quickly  to 
changes  in  actual  reliability.  Figures  4.2,  4.3,  4.4,  4.5,  and  4.6  demonstrate  the  model's 
performance  with  each  growth  pattern.  It  also  is  much  less  sensitive  to  the  selection  of 
failure  discounting  parameters  than  is  the  MLE  With  Discounting  Model.  However, 
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Figure  4.5     Reliability  Growth  Models 

High  Reliability  Moderate  Growth 

Std.  Discounting  Applied. 

the  variance  of  the  early  phase  estimates  is  much  greater  than  the  estimates  from  the 
MLE  With  Discounting  Model. 

This  model  requires  several  phases,  usually  at  least  four,  to  accurately  estimate 
system  reliability.  The  number  of  phases  required  is  a  function  of  the  shape  of  the 
reliability  growth  pattern.  More  phases  are  required  for  an  accurate  estimate  at  low 
system  reliability  than  at  high  reliability.  The  model  also  appears  to  track  changing 
reliability  more  accurately  than  constant  reliability. 

The  Exponential  Regression  Model  is  relatively  robust  with  respect  to  the 
selection  of  a  failure  discounting  method  and  parameters.  Still,  as  was  the  case  with  the 
MLE  With  Discounting  Model,  the  Lloyd  Discounting  Method  tends  to  cause  the 
model  to  overestimate  actual  reliability  as  is  demonstrated  by  Figure  4.11.  Figure  4.12 
demonstrates  how  robust  the  model  can  be  to  changing  parameters  when  the  Straight 
Percent   Discount    Method   is   applied.     Because   the    Straight    Percent    Discounting 
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Figure  4.6     Reliability  Growth  Models 

Low  Reliability  Rapid  Growth 

Std.  Discounting  Applied. 

Method  can  produce  nearly  equivalent  results  and  is  much  more  flexible,  use  of  the 
Lloyd  Discounting  Method  is  not  recommended  with  this  model. 

Again,  no  fixed  rules  exist  for  chosing  the  failure  discounting  parameters  of  the 
Straight  Percent  Discounting  Method.  Experience  with  the  model,  engineering 
judgement  and  possibly  computer  simulation  will  be  necessary  to  select  the  proper 
combination  of  the  discount  interval  and  discount  fraction.  Even  though  the  model  is 
fairly  robust  concerning  this  selection,  it  is  possible  to  choose  parameters  that  will 
force  the  model  to  overestimate  reliability.  The  danger  of  this  occurring  is  greatest 
when  expected  growth  in  the  system  reliability  does  not  occur. 

The  reliability  estimates  produced  by  the  exponential  regression  model  exhibit  a 
higher  variance  than  those  from  the  MLE  With  Discounting  Model.  This  increase  in 
variance  is  most  pronounced  in  the  early  testing  phases.  Figures  4.13  and  4.14  are  box 
plots  of  the  one-hundred  reliability  estimates  at  each  phase.  They  demonstrate  the 
variability  of  the  model. 
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Figure  4.7     MLE  With  Discounting 
Lloyd  Discounting  Method. 


As  with  any  regression,  several  points  are  required  to  stabilize  the  regression 
parameter  estimates.  This  causes  highly  variable  responses  in  the  first  two  or  three 
phases  of  testing.  This  variance  is  largest  when  reliability  in  the  early  phases  is  low.  In 
later  phases  and  at  higher  reliability,  the  variance  of  estimates  from-  the  Exponential 
Regression  Model  compares  favorably  to  those  of  the  MLE  With  Discounting  Model. 

The  Exponential  Regression  Model  closely  tracks  most  reliability  growth 
patterns.  It  is  best  when  more  than  three  test  phases  are  planned.  The  most  accurate 
results  were  obtained  when  the  model  was  used  with  the  Straight  Percent  Discounting 
Method.  The  modei  is  relatively  robust  with  respect  to  the  selection  of  discounting 
parameters;  but,  may  still  over  or  under  estimate  actual  reliability  if  a  bad  combination 
of  parameters  is  selected.  The  variance  of  reliability  estimates  in  the  early  testing 
phases  is  large  and  makes  the  model  undesirable  for  use  with  less  than  four  testing 
phases. 
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Figure  4.8     MLE  With  Discounting 
Straisht  Percent  Discounting  Method. 
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Figure  4.9     MLE  With  Discounting 
Estimate  Variability  at  Low  Constant  Reliability. 
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Figure  4.10    MLE  With  Discounting 
Estimate  Variability  With  Increasing  Reliability. 
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Figure  4.11     Exponential  Regression  Model 
Lloyd  Discounting  Method. 
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Figure  4.12    Exponential  Regression  Model 
Straight  Percent  Discounting  Method. 


60 


I—     & 

uj    o 


CD 

< 


EXPONENTIAL  REGRESSION  EST 
STD  DISCOUNTING  N=10.  R  =  .25 


- 

1 

[ 

: 

I 

: 

: 

: 

: 

5 

( 

! 

: 

f 

1 

¥ 

: 

t 

( 

"O- 

1 

-•<r" 

.  - 

..©.. 

1 

■o- 
X 

I, 

L 

PHASE  NUMBER 


10 


Figure  4.13     Exponential  Regression  Model 
Estimate  Variability  at  Low  Constant  Reliability. 
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Figure  4.14    Exponential  Regression  Model 
Estimate  Variability  With  Increasing  Reliability. 
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V.  SUMMARY  AND  CONCLUSIONS 

A.       SUMMARY 

The  purpose  of  this  study  is  to  develop  and  evaluate  discrete  reliability  growth 
models  that  employ  fractional  failure  reduction,  referred  to  as  failure  discounting. 
These  models  use  all  available  data  to  estimate  current  reliability  even  if  the  actual 
reliability  underlying  all  test  data  is  not  constant.  This  reduces  the  resources  required 
to  verify  system  reliability.  This  study  focuses  on  developmental  testing  scenarios  where 
testing  is  terminated  after  a  fixed  number  of  failures  have  been  observed  and  the  results 
of  each  trial  are  recorded  as  a  success  or  failure,  i.e.  attribute  data. 

Failure  discounting  is  performed  on  past  data  to  make  it  compatible  with  test 
data  taken  from  the  current  system  configuration.  Most  reliability  estimators  require  a 
constant  actual  reliability  for  each  trial  to  render  an  accurate  estimate.  Fractionally 
discounting  past  failures  adjusts  test  data  from  previous  testing  on  less  reliable  system 
configurations  making  the  data  useful  in  the  computation  of  the  current  reliability 
estimate. 

Two  different  failure  discounting  methods  were  used  in  the  study.  The  first, 
referred  to  as  the  Straight  Percent  Discounting  Method  allows  the  user  to  predetermine 
the  fraction  by  which  each  failure  will  be  reduced,  the  discount  fraction,  and  an  interval 
between  applications  of  the  discount  fraction,  the  discount  interval.  The  discount 
interval  is  the  number  of  trials  without  a  reoccurrence  of  the  particular  failure  cause 
between  application  of  the  discount  fraction.  The  second  failure  discounting  method, 
referred  to  as  the  Lloyd  Discounting  Method,  sets  all  past  failures  equal  to  the  upper 
confidence  bound  for  the  probability  of  reoccurrence  for  the  failure  cause.  Both  failure 
discounting  methods  were  used  with  each  of  the  three  discrete  reliability  growth  models 
developed  in  the  study. 

Three  discrete  reliability  growth  models  using  failure  discounting  were  developed. 
The  first  model  adds  failure  discounting  to  the  maximum  likelihood  estimate  for  a 
proportion.  The  model  is  referred  to  as  the  Maximum  Likelihood  Estimate  With 
Discounting  Model.  To  use  the  model,  the  failure  discounting  method  of  choice  is 
applied  to  the  test  data.  The  resulting  discounted  failures  are  returned  to  unity  by 
adjusting  the  number  of  successful  trials  preceding  each  failure.  The  estimate  of  system 


63 


reliability  is  computed  as  the  ratio  of  adjusted  successful  trials  to  adjusted  total  trials. 
Failure  discounting  allows  the  model  to  track  changing  reliability. 

The  remaining  two  models  are  modifications  of  an  exponential  single  phase 
reliability  estimate  of  the  form  reliability  =  1  —  e  .  One  model  uses  regression  to 
estimate  A.  This  model  is  referred  to  as  the  Exponential  Regression  Model.  The  second 
model  uses  an  unbiased  estimate  of  A  to  produce  a  single  testing  phase  reliability 
estimate.  The  current  reliability  estimate  is  a  weighted  average  of  the  previous  testing 
phase  reliability  estimate  and  the  current  single  phase  reliability  estimate.  The  fraction 
of  the  total  test  trials  to  date  used  in  the  computation  of  each  estimate  becomes  the 
weighting  factor  for  the  arithematic  average.  As  with  the  Maximum  Liklihood  With 
Discounting  Model,  a  failure  discounting  method  is  applied  to  the  test  data  before 
either  model  is  used  to  compute  a  reliability  estimate.  Both  these  models  have  the 
ability  to  track  changing  reliability  without  the  use  of  failure  discounting;  but,  failure 
discounting  is  employed  to  enhance  the  models  performance. 

A  Monte-Carlo  simulation  was  written  to  evaluate  the  performance  of  the 
reliability  growth  models  and  failure  discounting  methods  developed  in  the  study.  This 
was  necessary  because  closed  form  solutions  for  the  statistical  properties  of  the  models 
are  not  mathematically  tractable.  The  simulation  uses  input  parameters  to  generate  a 
reliability  growth  pattern.  Test  data  including  the  number  of  trials  up  to  and  including 
system  failure  and  the  cause  of  system  failure  are  generated  from  the  established 
reliability  growth  pattern.  Each  of  the  three  reliability  growth  are  applied  to  the 
simulated  test  data  resulting  in  an  estimate  of  system  reliability  at  each  testing  phase. 
After  this  procedure  is  replicated  a  suitable  number  of  times,  estimates  for  the  mean 
and  standard  deviation  of  each  estimate  are  computed.  These  estimates  are  used  to 
evaluate  the  performance  of  the  reliability  growth  models.  By  varying  the  reliability 
growth  pattern  and  the  failure  discounting  parameters,  the  models  performance  over  a 
wide  range  of  situations  may  be  evaluated.  The  conclusions  reached  from  this  analysis 
are  included  in  the  following  section. 

B.       CONCLUSIONS 

Even  though  the  analysis  of  the  reliability  growth  models  performed  in  the  study 
is  not  comprehensive  enough  to  state  conclusions  with  certainty,  several  general 
conclusions  are  apparent. 

The  Lloyd  Failure  Discounting  Method  should  not  be  used  with  the  three 
reliability  growth  models  developed  in  the  study.  With  few  exceptions,  this  method 
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caused  the  growth  models  to  overestimate  system  reliability.  This  tendency  to 
overestimate  was  particularly  evident  at  low  reliabilities  when  little  or  no  actual 
reliability  growth  occurred.  The  few  situations  where  the  method  performed  well,  rapid 
reliability  growth,  were  handled  equally  well  by  the  Straight  Percent  Discounting 
Method.  The  Straight  Percent  Discounting  Method's  flexibility  makes  the  preferred  of 
the  two  methods  for  fractionally  discounting  failures. 

The  Weighted  Average  Reliability  Growth  Model,  in  its  current  form,  should  not 
be  used  to  estimate  system  reliability.  The  model  grossly  overestimated  reliability  for 
most  growth  patterns  tested.  Even  when  no  failure  discounting  was  applied,  the  model 
overestimated  system  reliability.  Adding  either  failure  discounting  method  only 
increased  the  tendency  to  overestimate. 

Both  the  Maximum  Liklihood  Estimate  With  Discounting  and  the  Exponential 
Regression  Model  exhibited  performance  superior  to  the  standard,  single  phase  MLE, 
estimate.  But  there  is  no  clear  cut  choice  between  the  Maximum  Likelihood  Estimate 
With  Failure  Discounting  Model  and  the  Exponential  Regression  Model.  Both  models 
accurately  track  a  -wide  range  of  reliability  growth  patterns.  The  MLE  With 
Discounting  Model  appears  to  be  the  less  variable  of  the  two  methods  but  is  much 
more  sensitive  to  the  choice  of  failure  discounting  parameters  than  is  the  Exponential 
Regression  Model.  The  variability  of  both  models  decreased  with  increasing  reliability 
and  additional  test  data.  -For  this  reason  the  Exponential  Regression  Model  is 
recommended  for  situations  where  four  or  more  testing  phases  are  planned  and  the 
system  possesses  high  reliability  with  near  certainty.  For  other  cases  the  Maximum 
Likelihood  Estimate  With  Failure  Discounting  is  recommended  because  of  its  reduced 
variability. 

The  discrete  reliability  growth  models  using  failure  discounting  developed  in  the 
study  represent  an  improvement  over  conventional  reliability  estimation  techniques  for 
the  special  case  of  testing  until  a  fixed  number  of  failures  have  been  observed  with 
attribute  data.  However,  these  models  can  be  improved.  The  following  section 
contains  recomendations  for  improvements  and  further  study. 

C.       RECOMENDATIONS 

The  following  are  recomendations  for  improvement  to  the  models  introduced  in 

this  study. 

•  Many  more  simulation  runs  with  varied  reliability  growth  patterns  and  failure 
discounting  parameters  are  required  before  solid  conclusions  concerning  the 
properties  of  the  reliability  growth  models  can  be  reached. 
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The  Monte-Carlo  Simulation  should  be  modified  so  declining  reliability  can  be 
modeled. 

An  alternate  weighting  technique  should  be  developed  for  the  Weighted 
Average  Model  to  correct  its  tendency  to  overestimate  reliability.  In  certain 
situations,  this  model  demonstrated  the  smallest  variability  of  the  three  models 
developed.  The  model  might  prove  to  be  an  excellent  reliability  estimator  if  the 
overestimation  tendency  can  be  corrected. 

An  additional  failure  discounting  method  that  combines  the  flexibility  of  the 
Straight  Percent  Discounting  Method  with  the  non-arbitrary  selection  of  the 
discounting  fraction  from  the  Lloyd  method  might  prove  useful.  This  could  be 
accomplished  by  adding  a  discount  interval  to  the  current  Lloyd  Discounting 
Method. 

The  possibility  of  using  a  simulation  similar  to  the  one  developed  here  to  aid  in 
the  selection  of  failure  discounting  parameters  should  be  explored.  Using  an 
estimate  of  initial  reliability,  a  reliability  goal  and  the  number  of  planned  trials; 
the  various  choices  for  failure  discounting  parameters  could  be  tested  and  one 
set  chosen  based  on  the  simulation  results. 

Further  analysis  of  the  performance  of  the  reliability  growth  models  with 
varying  failure  discounting  parameters  is  required  to  develop  "rules  of  thumb" 
for  failure  parameter  selection. 
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APPENDIX  A 
DISCRETE  RELIABILITY  GROWTH  MODEL  SIMULATION 


1.        FORTRAN  CODE  FOR  THE  SIMULATION 


* 

X 

* 

* 

* 
* 

* 

* 
* 

* 

* 
* 

* 

* 
* 

* 

* 
* 
* 

* 
* 

* 


DISCRETE  RELIABILITY  GROWTH  SIMULATION 

PROGRAMMED  BY  JAMES  E  DRAKE 

LAST  MODIFIED  19  AUG  1987 

THE  FOLLOWING  EXTERNAL  FILES  ARE  USED  BY  THE  PROGRAM 
INPUT  :  DATA  AND  PARAMETER  INPUT  FILE  (DEVICE  #  10) 
THESIS  :  OUTPUT  FILE  CONTAINING  INTERMEDIATE  COMPUTATIONS 

(DEVICE  #  20) 
RELIAB:  OUTPUT  FILE  CONTAINING  FINAL  RESULTS  OF  THE  SIMULATION 

(DEVICE  #  30) 
EST     :  OUTPUT  FILE  CONTAINING  EACH  PHASE  ESTIMATE  FOR  EACH 

REPLICATION  OF  THE  WEIGHTED  AVERAGE  ESTIMATE 

(DEVICE  #  40) 
MLEWD   :  OUTPUT  FILE  CONTAINING  MLE  ESTIMATES  USING  DISCOUNTING 

FOR  EACH  PHASE  AND  EACH  REPLICATION 

(DEVICE  #  50) 
MLESP   :  OUTPUT  FILE  CONTAINING  MLE  ESTIMATE  FOR  EACH  SINGLE  PHASE 

AND  ALL  REPLICATIONS  USING  NO  DISCOUNTING 

(DEVICE  #  60) 
REGEST  :  OUTPUT  FILE  CONTAINING  EACH  PHASE  ESTIMATE  FOR  EACH 

REPLICATION  OF  THE  EXPONENTIAL  REGRESSION  ESTIMATE 

(DEVICE  #  70) 

THE  FOLLOWING  IS  A  LIST  OF  KEY  ARRAYS  USED  IN  THE  SIMULATION 

A      :  MAIN  WORKING  ARRAY  CONTAINS  PROBABILITY  OF  SUCCESS  FOR 

EACH  FAILURE  CAUSE,  NUMBER  OF  TRIALS  UNTIL  FAILURE  FOR 

EACH  FAILURE  CAUSE  AND  THE  SYSTEM,  CAUSE  OF  FAILURE, 

PHASE  NUMBER,  ADJUSTED  NUMBER  OF  TRIALS  AND  ADJUSTED 

NUMBER  OF  FAILURES 

DIMENSION  (  ((2*#CAUSES)+7),#FAILURES  ) 
NFAPH   :  CONTAINS  THE  NUMBER  OF  FAILURES  IN  EACH  PHASE 

DIMENSION  (1,#PHASES) 
NFCAUS  :  3INARY  ARRAY  USED  TO  DETERMINE  IF  A  FAILURE  OCCURRED  IN 

A  PHASE 

DIMENSION  (  1,#FAILURE  CAUSES) 
NTRIAL  :  CONTAINS  THE  NUMBER  OF  TRIALS  SINCE  LAST  FAILURE  OR 

DISCOUNTING  FOR  EACH  FAILURE  CAUSE 

DIMENSION  (  1,#FAILURE  CAUSES  ) 
PHREST  :  RECORDS  THE  PHASE  ESTIMATE  FOR  EACH  ESTIMATOR  WITHIN  A 

SINGLE  REPLICATION 

DIMENSION  (4,#PHASES) 

ROW  1  :  WEIGHTED  AVERAGE  ESTIMATE 

ROW  2  :  MLE  WITH  DISCOUNTING 

ROW  3  :  SINGLE  PHASE  MLE 

ROW  4  :  EXPONENTIAL  REGRESSION  ESTIMATE 

CONTAINS  ACTUAL  SYSTEM  RELIABILITY  IN  EACH  PHASE 

DIMENSION  (1,#PHASES) 

CONTAINS  YJK  VALUES  UP  TO  1000 

DIMENSION  (1,1000) 

CONTAINS  THE  NUMBER  OF  SUCCESS  AND  FAILURES  FOR  EACH 

FAILURE  CAUSE  (USED  WITH  WEIGHTED  AVERAGE  EST.) 

DIMENSION  (3,#FAILURE  CAUSES) 

ROW  1  :  NUMBER  OF  FAILURES 

ROW  2  :  NUMBER  OF  SUCCESSES 


AREL 
YJK 

CUMSF 


* 
* 
* 
* 
* 
* 

* 
* 
* 
* 
* 

* 
* 

* 
* 
* 
* 
* 
* 
* 
* 

* 

* 
* 
* 
* 
* 
* 


* 
* 

* 
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*  ROW  3  :  ADJUSTED  NUMBER  OF  SUCCESSES  * 

*  REG     :  ARRAY  USED  TO  COMPUTE  THE  EXPONENTIAL  REGRESSION  ESTIMATE   * 

*  DIMENSION  (5,#PHASES)  * 

*  ROW  1  :  K  BAR  * 

*  ROW  2  :  Y  BAR  * 

*  ROW  3  :  Y  BAR  FOR  THE  PHASE  * 

*  ROW  4  :  B  HAT  * 

*  ROW  5  :  A  HAT  * 

*  * 

*  THE  REMAINING  ARRAYS  ARE  USED  TO  COMPUTE  THE  MEAN  AND  VARIANCE  * 

*  OF  EACH  ESTIMATE  AT  EACH  PHASE.  THEY  ALL  HAVE  THE  SAME  DIMENSIONS    * 

*  AND  STRUCTURE  * 

*  DIMENSION  (4,#PHASES)  * 

*  ROW  1  :  RUNNING  SUM  OF  ESTIMATES  * 

*  ROW  2  :  RUNNING  SUM  OF  SQUARED  ESTIMATES  * 

*  ROW  3  :  MEAN  OF  THE  ESTIMATES  * 

*  ROW  4  :  STANDARD  DEVIATION  OF  THE  ESTIMATES  * 

*  * 

VALUES  FOR  THE  WEIGHTED  AVERAGE  ESTIMATE  * 

VALUES  FOR  THE  MLE  WITH  DISCOUNTING  * 

VALUES  FOR  THE  SINGLE  PHASE  MLE  * 

VALUES  FOR  THE  EXPONENTIAL  REGRESSION  ESTIMATE  * 

*********************************************** 


*  EST 

*  MLEWD 

*  MLESP 

*  REGEST 


C  DEFINE  AND  DIMENSION  VARIABLES 

PARAMETER  (NR=50 /NC=200) 

INTEGER  REP,CUMSF,DISOPT 

REAL*4  MIN 

REAL*S  DSEED, MLESP. MLEWD, EST 

DIMENSION  NFAPH(NRJ,A(NR,NC),NFCAUS(NR),NTRIAL(NR),PHREST(4,NR)/ES 
CT(4,NR) ,MLEWD(4,NR)/MLESP(4,NR),REGEST(4/NR) ,AREL(NR) , YJK(IOOO) ,CU 
CMSF(3,NR),REG(5,NR) 

C  READ  IN  THE  NUMBER  OF  CAUSES  TO  BE  USED  (  NCAUSE  )  AND  THE  NUMBER 
C  OF  PHASES  (  NPHASE  )  IN  THE  TEST 

READ (10,*)  NCAUSE 
READ (10,*)  NPHASE 

C   CREATE  VARIABLES  FOR  THE  ROW  INDICES  OF  THE  WORKING  MATRIX  (  A  ) 
C     IPHASE •  PHASE 

c   isyspr!  actual  component  reliability 

C  INTR:  NUMBER  OF  TRIALS  UP  TO  AND  INCLUDING  FAILURE 

C  IFAILC:  CAUSE  OF  THE  FAILURE 

C  IADJF:  ADJUSTED  NUMBER  OF  FAILURES  ED 

C  AFTER  DISCOUNTING  HAS  BEEN  APPLIED 

C  IADJT:  ADJUSTED  NUMBER  OF  TRIALS  AFTER  DISCOUNTING  HAS  BEEN  APPLIED 

C  IYJK:  YJK  COMPUTED  ON  THE  ADJUSTED  NUMBER  OF  TRIALS 

IPHASE  =  (2*NCAUSE)+1 
ISYSPR  =  IPHASE  +1 
INTR  =  ISYSPR  +  1 
IFAILC  =  INTR  +  1 
IADJF  =  IFAILC  +  1 
IADJT  =  IADJF  +  1 
IYJK  =  IADJT  +  1 

C   READ  IN  THE  NUMBER  OF  FAILURES  IN  EACH  PHASE  (  NFAPH(I)  )  AND 
C   COMPUTE  THE  TOTAL  NUMBER  OF  FAILURES  IN  THE  TEST  (  NFAIL  ) 

NFAIL  =  0 

DO  10  1=1, NPHASE 

READ(10,*)  NFAPH(I) 

NFAIL  =  NFAIL  +  NFAPH(I) 
10    CONTINUE 

C   INPUT  THE  PROBABILITY  OF  SUCCESS  IN  A  SINGLE  TRIAL  FOR  EACH  CAUSE 
C   IN  THE  FIRST  PHASE 
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DO  20  I=1,NCAUSE 

READ(10,*)  A(I,1) 
20   CONTINUE 

C  INPUT  THE  REMAINING  VARIABLES  ,  THE  NUMBER  OF  SUCCESSFUL  TRIALS 

C  BEFORE  A  DISCOUNT  IS  APPLIED  (N) ;  THE  DISCOUNT  FACTOR  (R) ;  THE  SEED 

C  FOR  THE  RANDOM  NUMBER  GENERATOR,  GGUBFS ,  (DSEED) ;  RELIABILITY 

C  GROWTH  FRACTION  (FRIMP);  TRIGGER  FOR  PRINTING  INTERMEDIATE  OUTPUT 

C  (IOPT) 

C  TRIGGERS  FOR  SAVING  EACH  ESTIMATE  AT  EACH  PHASE  FOR  EACH  ESTIMATOR 

C      IOPT1  :  WEIGHTED  AVERAGE  MODEL 

C      IOPT2  :  MLE  WITH  DISCOUNTING 

C      IOPT3  :  SINGLE  PHASE  MLE 

C      I0PT4  :  EXPONENTIAL  REGRESSION  MODEL 

C  DISCOUNTING  OPTION  TRIGGER  (DISOPT);  LLOYDS  FAILURE  DISCOUNTING 

C  PARAMETER  (GAMMA) 

READ(10,*)  N 
READ (10,*)  R 
READ(10,*)  DSEED 
READ (10,*)  FRIMP 
READ(10,*)  NREP 
READ (10,*)  IOPT 
READ(10,*)  IOPT1 
READ (10,*   IOPT2 
READ (10,*   IOPT3 
READ (10,*)  I0PT4 
READ(10,*)  DISOPT 
READ (10,*)  GAMA 

XNREP  =  NREP 
DSEED1  =  DSEED 

C   INITIALIZE  THE  ARRAYS  USED  TO  COMPUTE  THE  MEAN  AND  STANDARD  DEVIATION 
C   OF  EACH  ESTIMATOR 

DO  30  J=1,NPHASE 
DO  30  1=1,4 

EST(I.,J)  =  0.0 
MLEWD(I,J)  =0.0 
MLESP(I,J)  =0.0 
REGEST(I,J)  =0.0 
PHREST(I,J)  =0.0 
30   CONTINUE 

C  COMPUTE  AND  STORE  THE  YJK  VALUES  UP  TO  1000 

YJK(l)  =0.0 
DO  40  1=1,999 

YJK(I+1)  =  YJK(I)  +  1.0/1 
40   CONTINUE 

C  COMPUTE  AND  STORE  K  BAR  FOR  THE  EXPONENTIAL  REGRESSION  MODEL 

SUM  =  0.0 
DO  50  I=1,NPHASE 
SUM  =  SUM  +  I 
REG(1,I)  =  SUM/I 
50    CONTINUE 

C  MAJOR  REPETITION  OF  THE  SIMULATION  LOOP 

DO  500  REP=1,NREP 

C   INITIALIZE  FAILURE  CAUSE  VECTOR  (NFCAUS)  AND  (CUMSF) 
C   COMPUTE  THE  INITIAL  SYSTEM  RELIABILITY 

REL  =  1. 

DO  60  I=1,NCAUSE 

NFCAUS(I)  =  0 

REL  =  REL  *  A(I,1) 

DO  60  J=1.3 

CUMSF (J, I)  =  0 
60    CONTINUE 
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C   INITIALIZE  COLUMN  (FAILURE  #  )  COUNTER  FOR  THE  WORKING  ARRAY  (A) 

J  =  1 

C  LOOP  TO  COMPUTE  THE  NUMBER  OF  TRIALS  UP  TO  AND  INCLUDING  FAILURE 
C  AND  THE  CAUSE  OF  FAILURE  FOR  EACH  FAILURE  IN  EACH  PHASE 

DO  130  K=1,NPHASE 

C   SKIP  ACTUAL  COMPONENT  RELIABILITY  COMPUTATION  AFTER  FIRST  REP 
C  AND  FOR  FIRST  FAILURE 

IF(J.EQ.l)  GOTO  75 
IF(REP.GT.l)  GOTO  75 
REL  =  1. 

C   COMPUTE  NEW  ACTUAL  RELIABILITY  FOR  THE  COMPONENT  IN  PHASE  K 

DO  70  I=1,NCAUSE 

C   INCREASE  CAUSE  PR (SUCCESS)  IF  IT  CAUSED  FAILURE  IN  THE  PREVIOUS  PHASE 
C   COMPUTE  NEXT  PHASE  RELIABILITY  AND  REINITIALIZE  NFCAUS 

IF (NFCAUS ( I ) . EO . 1 )  THEN 

A(I,J)  =  A(I,(J-1))  +  ((1.  -  A(I,(J-1)))*FRIMP) 

ELSEIF(NFCAUS(I) .NE.l)  THEN 

A(I,J)  =  A(I,(J-1)) 

ELSE 

END  IF 

REL  =  REL*A(I,J) 

NFCAUS (I)  =  0 
70      CONTINUE 
75     Jl  =  1 

TRTOT  =0.0 

C   COMPUTE  THE  NUMBER  OF  TRIALS  UP  TO  AND  INCLUDING  FAILURE  AND  THE 
C   CAUSE  OF  FAILURE  FOR  EACH  FAILURE  IN  THE  PHASE 

DO  120  L=1,NFAPH(K) 

IF(REP.GT.l)  GOTO  90 

IF(Jl.EO.l)  GOTO  35 

DO  80  I=1,NCAUSE 

A(I,J)  =  A(I,(J-1)) 
80       CONTINUE 
85       A(ISYSPR,J)  =  REL 

A(IPHASE,J)  =  K 
90       MIN  =  7.2E75 

DO  110  I=1,NCAUSE 

C  ASSIGN  #  TRIALS  FOR  CAUSES  WITH  PR (SUCCESS)  =  0  OR  1 

IF(A(I,J).GE.l.)  THEN 

A((I+NCAUSE) ,J)  =  7.2E75 

GOTO  100 
ELSEIF(A(I,J) .EQ.O.)  THEN 

A((I+NCAUSE) ,J)  =  1. 

GOTO  100 
ELSE 
END  IF 

C  CONVERT  UNIFORM  (0,1)  RANDOM  VARIABLE  TO  GEOMETRIC  (#  TRIALS  UNTIL 

C  FAILURE  )  FOR  EACH  FAILURE  CAUSE.  RECORD  THE  MIN  #  TRIALS  FOR  THE 

C  CAUSES  AS  THE  SYSTEM  #  TRIALS  UP  TO  AND  INCLUDING  FAILURE  AND 

C  RECORD  THE  FAILURE  CAUSE 

A((I+NCAUSE) ,J)  =  INT(l.+(LOG(GGUBFS(DSEED))/LOG(A(I,J)))) 
100        IF(A((I+NCAUSE) ,J).LE.MIN)  THEN 
MIN  =  A((I+NCAUSE),J) 
IMIN  =  I 
ELSE 
END  IF 
110      CONTINUE 

A(IFAILC,J)  =  IMIN 
NFCAUS (IMIN)  =  1 

C   COMPUTE  THE  TOTAL  #  OF  TRIALS  FOR  THE  MLE  SINGLE  PHASE  ESTIMATE  AND 
C   INCREMENT  FAILURE  #  COUNTERS 
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A(INTR,J)  =  MIN 
TRTOT  =  TRTOT  +  A(INTR,J) 
J  =  J  +  1 
Jl  =  Jl  +  1 
120   CONTINUE 

C   COMPUTE  THE  MLE  ESTIMATE  OF  COMPONENT  RELIABILITY  FOR  THIS  PHASE  AND 
C   COMPUTE  THE  RUNNING  SUM  OF  ESTIMATES  AND  THE  SUM  OF  ESTIMATES  SQUARED 
C   FOR  COMPUTATION  OF  THE  MEAN  AND  STANDARD  DEVIATION  OF  THE  ESTIMATE 


PHREST(3,K)  =  (TRTOT  -  NFAPH(K) ) /TRTOT 
MLESP(1,K)  =  MLESP(1,K)  +  PHREST(3,K) 
MLESP(2,K)  =  MLESP(2,K)  +  (PHREST (3 ,K)**2) 


130   CONTINUE 

C   INITIALIZE  THE  ADJUSTED  NUMBER  OF  FAILURES  TO  1  AND  THE  COUNT  OF  THE 
C  NUMBER  OF  TRIALS  SINCE  FAILURE  OR  DISCOUNTING  (NTRIALS(I)  )  TO  0 
C   IN  PREPARATION  FOR  THE  DISCOUNTING  ROUTINE 

DO  140  J=1,NFAIL 

A(IADJF,J)  =  1. 
140   CONTINUE 

DO  150  I=1,NCAUSE 

NTRIAL(I)  =  0 
150   CONTINUE 

C   DISCOUNTING  ROUTINE  REVIEWS  ALL  PAST  FAILURES  AND  CAUSES  TO  DATE 

C  AND  DETERMINES  IF  THE  DISCOUNTING  CONDITIONS  HAVE  BEEN  MET.  COMPUTES 

C  THE  ADJUSTED  FAILURES,  THE  ADJUSTED  #  OF  TRIALS  AND  YJK 

J  =  0 

DO  300  K=1,NPHASE 

DO  200  L=1,NFAPH(K) 
J  =  J  +  1 

C  UPDATES  THE  NUMBER  OF  TRIALS  SINCE  FAILURE  OR  DISCOUNTING  FOR  EACH 
C   FAILURE  CAUSE 

ICAUSE  =  TNT(A(IFAILC,J)+.5) 
DO  160  I=1,NCAUSE 

IF ( ICAUSE. EQ. I)  THEN 

NTRIAL(I)  =  0 
ELSE I F< ; ICAUSE. NE . I)  THEN 

NTRIAL(I)  =  NTRIAL(I)  +  INT(A(INTR, J)+. 5) 
ELSE 
END  IF 
160     CONTINUE 

200   CONTINUE 

C   CHOOSE  DISCOUNTING  METHOD  TO  BE  USED 

IF(DISOPT.NE.2)  GOTO  130 

C   PERFORM  LLOYDS  FAILURE  DISCOUNTING  METHOD 

DO  170  1=1, J 

II  =  INT(A(IFAILC,I)+.5) 
IF(NTRIAL(I1).EQ.0)  THEN 

A(IADJF,I)  =1.0 

GOTO  170 
ELSE 
END  IF 

A(IADJF,I)  =  1.0  -  ((l.-GAMA)**(1.0/NTRIAL(Il))) 
170     CONTINUE 

GOTO  210 

C  PERFORMS  STRAIGHT  PERCENT  FAILURE  DISCOUNTING  AND 
C   COMPUTES  THE  ADJUSTED  #  OF  FAILURES 

180      DO  190' 1=1, J 

II  =  INT(A(IFAILC,I)+.5) 
IF (NTRI AL ( I 1 ) . EQ . 0 )  THEN 

A(IADJF,I)  =  1. 
ELSEIF(NTRIAL(I1).GE.N)  THEN 
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A(IADJF,I)  =  A(IADJF,I)*((1.-R)**(NTRIAL(I1)/N)) 

ELSE 
END  IF 
190      CONTINUE 

C  ADJUSTS  THE  #  TRIALS  SINCE  FAILURE  OR  DISCOUNTING  FOR  THOSE  CAUSES 
C  THAT  HAVE  MET  OR  SURPASSED  THE  DISCOUNTING  THRESHOLD 
C   FOR  THE  STRAIGHT  PERCENT  DISCOUNTING  METHOD 

DO  205  I=1,NCAUSE 

IF(NTRIAL(I).GE.N)  NTRIAL(I)  =  MOD(NTRIAL(I ) ,N) 
205      CONTINUE 
210    TADJT  =0.0 
TYJK  =0.0 
TPYJK  =0.0 
Kl  =  0 

DO  215  12=1,3 
DO  215  I=1,NCAUSE 
CUMSF(I2,I)  =  0 
215       CONTINUE 

C   COMPUTES  THE  ADJUSTED  #  OF  TRIALS  FROM  THE  ADJUSTED  #  OF  FAILURES 
C  AND  COMPUTES  THE  SUM  OF  THE  ADJUSTED  #  OF  TRIALS  FOR  ESTIMATE  COMP . 

PREL  =0.0 
LTRIAL  =  0 

DO  240  1=1, J 

A(IADJT,I)  =  A(INTR,I)/A(IADJF,I) 
TADJT  =  TADJT  +  A(IADJT,I) 

C   COMPUTE  YJK  FROM  THE  ADJUSTED  #  OF  TRIALS  AND  STORE  THE  SUM  FOR 

C   ESTIMATE  COMPUTATION,  USE  ARRAY  FOR  #  TRIALS  <  1000  AND  APPROX.  FOR 

C  VALUES  >  1000 

Nl  =  NINT(A(IADJT,I)) 
'  IF(Nl.LE.lOOO)  THEN 

•  A(IYJK,I)  =  YJK(Nl) 
ELSEIF(Nl.GT.lOOO)  THEN 

A(IYJK,I)  =  (A(IADJT, I)**. 10995)*3 . 55445 
ELSE 
ENDIF 

C  DETERMINE  IF  A  PHASE  BOUNDARY  HAS  BEEN  REACHED  TO  BEGIN  ESTIMATE 
C   COMPUTATION 

IF(I.EQ.l)  GOTO  225 
IF(A(IPHASE,I).NE.A(IPHASE,(I-1)))  THEN 

C   COMPUTE  THE  WEIGHTED  AVERAGE  ESTIMATE 

MAX  =  0 

Kl  =  Kl  +  1 

C  DETERMINE  THE  FAILURE  CAUSE  WITH  THE  LARGEST  #  OF  FAILURES 

DO  220  I1=1,NCAUSE 

IF(CUMSF(1,I1).GT.MAX)  THEN 
MAX  =  CUMSF(1,I1) 
ICOL  =  II 

ELSE 
ENDIF 
220   CONTINUE 

C  COMPUTE  YJK  VALUE  FOR  THE  CURRENT  PHASE  ESTIMATE 

IF(CUMSF(1, ICOL). LE. 1000)  THEN 

AHATL  =  YJK(CUMSF(l,ICOL)) 
ELSEIF(CUMSF(l,ICOL) .GT.1000)  THEN 

X  =  CUMSF(l.ICOL) 

AHATL  =  (X**.10995)*3. 55445 
ELSE 
ENDIF 

IX  =  CUMSF(l,ICOL)  +  CUMSF(3,ICOL) 
IF(IX.LE.IOOO)  THEN 
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AHATU  =  YJK(IX) 
ELSEIF(IX.GT.IOOO)  THEN 

X  =  IX 

AHATU  =  (X**. 10995)*3. 55445 
ELSE 
END  IF 

C  COMPUTE  CURRENT  PHASE  RELIABILITY  ESTIMATE 

AHAT  =  AHATU  -  AHATL 

CREL  =  1.0  -  EXP ( -AHAT) 

X  =  CUMSF (1, ICOL)  +  CUMSF(3,ICOL) 

C   COMPUTE  AND  STORE  THE  WEIGHTED  AVERAGE  ESTIMATE 

PREL  =  ((LTRIAL*PREL)/X)  +  ( ( (X-LTRIAL)*CREL)/X) 
LTRIAL  =  CUMSF(l,ICOL)  +  CUMSF(3 , ICOL) 

C   COMPUTE  THE  PHASE  AND  GLOBAL  AVERAGE  FOR  YJK  USED  IN  THE  WOODS 
C   REGRESSION  ESTIMATES  ARE 

REG(2,K1)  =  TYJK/(I-1) 
REG(3,K1)  =  TPYJK/NFAPH(K1) 
TPYJK  =0.0 

END  IF 

C   COMPUTE  THE  NUMBER  OF  FAILURES  AND  SUCCESSES  FOR  EACH  FAILURE  CAUSE 
C   USED  IN  THE  WEIGHTED  AVERAGE  ESTIMATE 

225   ICAUSE  =  INT(A(IFAILC,I)+.5) 
DO  230  I1=1,NCAUSE 

CUMSF(2,I1)  =  CUMSF(2,I1)  +  INT(A(INTR, I )  +  .5) 
CUMSF(3,I1)  =  CUMSF(3,I1)  +  Nl 
230   CONTINUE 

CUMSF (1,1 CAUSE)  =  CUMSF(1 , ICAUSE' 

CUMSF (2, I CAUSE)  =  CUMSF (2 , ICAUSE' 

CUMSF(3, ICAUSE)  =  CUMSF (3 , ICAUSE' 

TPYJK  =  TPYJK  +  A(IYJK.I) 

TYJK  =  TYJK  +  A(IYJK,I) 

240    CONTINUE 

C  REPEAT  COMPUTATIONS  FOR  THE  WEIGHTED  AVERAGE  ESTIMATE  FOR  THE 
C   FINAL  PHASE 

MAX  =  0 

Kl  =  Kl  +  1 

DO  245  I1=1,NCAUSE 

IF(CUMSF(1,I1).GT.MAX)  THEN 

MAX  =  CUMSF(1,I1) 

ICOL  =  II 

ELSE 

END  IF 
245   CONTINUE 

IF(CUMSF(1, ICOL). LE. 1000)  THEN 

AHATL  =  YJK(CUMSF(l,ICOL)) 
ELSEIF ( CUMSF ( 1 , ICOL ) . GT . 1000 )  THEN 

X  =  CUMSF(l.ICOL) 

AHATL  =  (X**. 10995)*3. 55445 
ELSE 
END  IF 

IX  =  CUMSF(l,ICOL)  +  CUMSF(3/ICOL) 
IF(IX.LE.IOOO)  THEN 

AHATU  =  YJK (IX) 
ELSEIF(IX.GT.IOOO)  THEN 

X  =  IX 

AHATU  =  (X**. 10995)*3. 55445 
ELSE 
ENDIF 

AHAT  =  AHATU  -  AHATL 
CREL  =  1.0  -  EXP (-AHAT) 

X  =  CUMSF(l,ICOL)  +  CUMSF(3,ICOL) 

PREL  =  ((LTRIAL*PREL)/X)  +  ( ( (X-LTRIAL)*CREL)/X) 
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LTRIAL  =  CUMSF(1,IC0L)  +  CUMSF(3 , ICOL) 
REG(2,K1)  =  TYJK/(J) 
REG(3,K1)  =  TPYJK/NFAPH(K1) 

PHREST(1,K)  =  PREL 
C   COMPUTE  THE  MLE  ESTIMATE  OF  PHASE  RELIABILITY  USING  DISCOUNTING 

PHREST(2,K)  =  (TADJT  -  J)/TADJT 

C   COMPUTE  THE  EXPONENTIAL  REGRESSION  ESTIMATE  BEGINNING  WITH  B  HAT 

SUM  =  0.0 

SUMS  =0.0 

IF  (K.EQ.l)  GOTO  252 

DO  250  I  =  1,K 

SUM  =  SUM  +  ((I-REG(1,K))*REG(3,I)) 

SUMS  =  SUMS  +  ((I-REG(1,K))**2) 
250     CONTINUE 

REG(4,K)  =  SUM/SUMS 

C   COMPUTE  A  HAT 

REG(5,K)  =  REG(2,K) 


-  (REG(4,K)*REG(1,K)) 


C   COMPUTE  AND  STORE  THE  EXPONENTIAL  REGRESSION  ESTIMATE 


(REG(4,K)*K))) 
0 


PHREST(4,K)  =  1.0  -  EXP(- (REG(5 ,K)  + 
IF(PHREST(4,K) .LT.0.0)  PHREST(4,K)=0 .i 
GOTO  255 
252     PHREST(4,K)  =  1.0  -  EXP(-REG(3 , 1) ) 

IF(PHREST(4,X) .LT.0.0)  ?HREST(4,K)=0 .0 

C  STORE  THE  RUNNING  SUM  OF  THE  ESTIMATES  FOR  THE  CURRENT  PHASE  AND  THE 

C  RUNNING  SUM  OF  THE  ESTIMATES  SQUARED  FOR  COMPUTATION  OF  THE  MEAN  AND 

C  STANDARD  DEVIATION  OF  EACH  ESTIMATE  FOR  EACH  RELIABILITY  GROWTH 

C  MODEL 

255    EST(1,K)  =  EST(1,K)  +  PHREST(1,K) 

EST(2,K)  =  EST(2,K)  +  (PHREST(1 ,K)**2) 
MLEWD(1,K)  =  MLEWD(1,K)  +  PHREST(2,K) 
MLEWD(2,K)  =  MLEWD(2,K)  +  (PHREST(2 ,K)**2) 
REGEST(1,K)  =  REGEST(1,K)  +  PHREST(4,K) 
REGEST(2,K)  =  REGEST(2,K)  +  (PHREST(4,K)**2) 

C   STORE  THE  ACTUAL  PHASE  RELIABILITY 

AREL(K)  =  A(ISYSPR,J) 

C  PRINT  INTERMEDIATE  OUTPUT  IF  REQUESTED  AND  THE  NUMBER  OF  REPETITIONS 
C   IS  NOT  GREATER  THAN  5 

IF(IOPT.NE.l)  GOTO  300 
IF (REP. GT. 5)  GOTO  300 

WRITE (20, 1000)  REP,K 
1000  FORMAT (T16, 'REPETITION  NUMBER:  \I4,' 

WRITE(20,1010)  A(ISYSPR,J) 
1010  FORMAT ( 2 2X, 'ACTUAL  COMPONENT  RELIABILITY:  \F7.5) 

WRITE(20,1020)  PHREST(1,K) 
1020  FORMAT (20X, 'PREDICTED  COMPONENT  RELIABILITY:  \F7.5) 

WRITE(20,1022)  PHREST(2,K) 
1022  FORMAT (2 OX, 'MLE  ESTIMATE  USING  DISCOUNTING:  ',F7.5) 

WRITE(20,1025)  PHREST(3,K) 
1025  FORMAT (18X, 'MLE  ESTIMATE  OF  PHASE  RELIABILITY:  \F7.5) 

WRITE(20,1027)  PHREST(4,K) 
1027  FORMAT (14X, 'REGRESSION  ESTIMATE  OF  PHASE  RELIABILITY: 

WRITE(20,1030) 
1030  FORMAT ( '  ' , '      ' ) 

DO  260  I=1,NCAUSE 

WRITE(20,1035)I,A(I,J),A((I+NCAUSE),J) 

1035  FORMAT (12X, 'CAUSE:  ',13,'    PR(SUCCESS) :  ',F7.6,' 
CF10.0) 

260   CONTINUE 

WRITE(20,1036) 

1036  FORMAT ('  ' , '      ' )  • 

WRITE(20,1040) 


PHASE  NUMBER: 


,14) 


F7.5) 


#  TRIALS: 
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1040  FORMAT (4X, 'FAIL  #',3X,'FAIL  CAUSE1, 3X,'#  TRIALS ' ;3X, ■ ADJ  #  FAIL' ,3 
CX,'ADJ  #  TRIALS' ^X/YJK' ) 
DO  270  1=1, J 

WRITE(20,1050)I,A(IFAILC,I),A(INTR,I),A(IADJF,I),A(IADJT,I),A(IYJK 
C,I) 
1050  F0RMAT(4X,  13  ,8X,F3 .0 ,7X,F8. 0 ,4X,F8.6 ,4X,F12.0 ,3X,F11 . 4) 
270   CONTINUE 

WRITE(20,1060) 
1060  F0RMAT( '  ',///) 

300   CONTINUE 
C  PRINT  EACH  OF  THE  3  ESTIMATES  TO  THEIR  APPROPRIATE  OUTPUT  FILE 
C   IF  REQUESTED 

IF(IOPTl.NE.l)  GOTO  401 

400  WRITE(40,2000)  (PHREST(1 , I) ,  I=1,NPHASE) 

401  IF(I0PT2.NE.l)  GOTO  402 

WRITE (50, 2000)  (PHREST(2 , I) ,  I=1,NPHASE) 

402  IF(I0PT3.NE.l)  GOTO  403 

WRITE(60,2000)  (PHREST(3 , I) ,  I=1,NPHASE) 

403  IF(I0PT4.NE.l)  GOTO  500 

WRITE(70,2000)  (PHREST(4, I) ,  1=1, NPHASE) 
2000FORMAT('  ' ,30(F7 .6 :1X) ) 

500   CONTINUE 

C   UPON  COMPLETION  OF  ALL  REPETITIONS,  COMPUTE  THE  MEAN  AND  STANDARD 
C   DEVIATION  OF  EACH  ESTIMATE  FOR  EACH  PHASE  SKIPPING  COMPUTATIONS  IF 
C   ONLY  ONE  REPETITION  IS  REQUIRED 

IF  (NREP.LE.l)  GOTO  601 

DO  600  I=1;NPHASE 

EST(3,I)  =  EST(1,I)/XNREP 

MLEWD(3,I)  =  MLEWD(1,I)/XNREP 

MLESP(3,I)  =  MLESP(1,I)/XNREP 

REGEST (3, I)  =  REGEST (1, I )/XNREP 

EST(4,I)  =  SQRT((EST(2,I)-(XNREP*(EST(3,I)**2)))/(XNREP-1)) 

MLEWD(4,I)  =  SQRT((MLEWD(2/I)-(XNREP*(MLEWD(3,I)*^2)))/(XNREP-1 


MLESP(4,I)  =  SQRT((MLESP(2,I)-(XNREP*(MLESP(3,I)**2)))/(XNREP-1 
REGEST (4 , I ) =SQRT ( (REGEST (2,1) - (XNREP* (REGEST ( 3 , I ) **2 ) ) ) / (XNREP- 1 ) ) 

600  CONTINUE 

C  PRINT  THE  FINAL  OUTPUT  TABLE  TO  A  FILE 

601  WRITE(30,3000) 

3000  FORMAT (' 0 ' ,T47  ' DISCRETE  RELIABILITY  GROWTH  SIMULATION') 

WRITE(30,3010) 
3010  FORMAT ('-' ,T54. 'MODEL  PARAMETER  SUMMARY') 

WRITE(30,3020)  NCAUSE 
3020  FORMAT (' 0 ', T47 .' NUMBER  OF  POSSIBLE  FAILURE  CAUSES  ',14) 

WRITE(30,3030) 
3030  FORMAT('0' ,T38, 'CAUSE  NUMBER' ,T64, ' SINGLE  TRIAL  PR(  SUCCESS  )  FOR 
CPHASE  1 ' ) 

DO  3050  M=l, NCAUSE 

WRITE(30,3040)  M,A(M,1) 
3040  FORMAT ('  ' ,T43 , 12 ,T79 ,F8.6) 
3050  CONTINUE 

WRITE (30, 3060)  FRIMP 
3060  FORMAT('0' ,T37, 'FRACTION  CAUSE  RELIABILITY  IMPROVES  AFTER  FAILURE 
C  ,F8.6) 

WRITE(30,3080)  NPHASE 
3080  F0RMAT('-' ,T43. 'NUMBER  OF  PHASES  IN  THE  SIMULATION  ',12) 

WRITE(30,3090) 
3090  F0RMAT( '0' ,T42, 'PHASE  NUMBER ' ,T59 , 'NUMBER  OF  FAILURES  IN  THE  FIRST 
C  PHASE1 ) 

DO  3110  M=l, NPHASE 

WRITE (30, 3100)  M,NFAPH(M) 
3100  F0RMAT('  ' ,T43 , 12 ,T73 , 12) 
3110  CONTINUE 

WRITE(30,3120)  NFAIL 
3120  FORMAT('0' ,T51, 'TOTAL  NUMBER  OF  FAILURES  ',14) 

IF(DISOPT.EQ.2)  GO  TO  3160 

WRITE(30,3130) 
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3130  FORMAT ('-' ,T38, 'DISCOUNTING  PERFORMED  USING  THE  CONSTANT  FRACTION 
CMETHOD ' ) 
WRITE(30,3140)  R 
3140  FORMAT('0'  ,T44.  'FRACTION  EACH  FAILURE  IS  DISCOUNTED  \F8.6) 

WRITE(30,3150)  N 
3150  FORMAT('  ' ,T33 , 'NUMBER  OF  TRIALS  AFTER  A  FAILURE  BEFORE  A  DISCOUNT 
C  IS  APPLIED  ' ,14) 
GO  TO  3190 
3160  WRITE(30,3170) 
3170  FORMAT('-' ,T44. 'DISCOUNTING  PERFORMED  USING  THE  LLOYD  METHOD') 

WRITE (30, 3 180)  GAMA 
3180  FORMAT ('0' ,T39, 'PERCENT  C.I.  (  USED  AS  DISCOUNT  FRACTION  )  ',F8.6 

C) 
3190  WRITE(30,3200)  DSEED1 
3200  FORMAT ('-' ,T46. 'RANDOM  NUMBER  SEED  USED  ',F15.2) 

WRITE(30,3210)  NREP 
3210  FORMAT( '0' ,T37, 'NUMBER  OF  REPETITIONS  OF  THE  SIMULATION  PERFORMED 
C',17) 
WRITE(30,3220) 
3220  FORMAT ('1' ,T61. 'ESTIMATOR: ' ) 

WRITE(30,3230) 
3230  FORMAT ('0.'  ,T48.  'SINGLE'  PHASE  MLE  WITHOUT  DISCOUNTING') 

WRITE(30,3240) 
3240  FORMAT ('-' ,T60. 'MEAN' ,T83, 'ESTIMATE' ,T109, '95  %' ) 

WRITE(30,3250) 
3250  FORMAT ('  ',T12,' PHASE  NUMBER' ,T29 ,' ACTUAL  RELIABILITY' ,T52 ,' PREDIC 
CTED  RELIABILITY' ,T78, 'STANDARD  DEVIATION ' ,T101 ,' CONFIDENCE  INTERVA 
CL1  ) 

C   COMPUTE  C.I.  FOR  SINGLE  PHASE  MLE 

DO  3270  M=1,NPHASE 

CI  =  (1.96*MLESP(4,M))/SQRT(XNREP) 

CIU  =  MLESP(3,M)  +  CI 

CIL  =  MLESP(3.M)  -  CI 

WRITE(30,3260)  M,AREL(M) ,MLESP(3 ,M) ,MLESP(4,M) , CIL, CIU 
3260  FORMAT('0' ,T17 , 12 ,T34,F8.6 ,T53 ,F8.6 ,T82 ,F9 .6 ,T99 , ' (  ',F8.6,'  ,  '  ,F 

C8.6,'  )') 
3270  CONTINUE 

WRITE(30,3220) 

WRITE(30,3280) 
3280  FORMAT('0' ,T42, 'MAX  LIKELIHOOD  ESTIMATE  USING  DISCOUNTED  FAILURES' 
C) 

WRITE(30,3240) 

WRITE(30,3250) 

C  COMPUTE  C.I.  FOR  MLE  WITH  DISCOUNTING 

DO  3290  M=1,NPHASE 

CI  =  (1.96*MLEWD(4,M))/SQRT(XNREP) 

CIU  =  MLEWD(3,M)  +  CI 

CIL  =  MLEWD(3.M)  -  CI 

WRITE(30,3260)  M,AREL(M) ,MLEWD(3,M) ,MLEWD(4 ,M) , CIL, CIU 
3290  CONTINUE 

WRITE(30,3220) 

WRITE(30,3300) 
3300  FORMAT('0' ,T38, 'WEIGHTED  AVERAGE  ESTIMATE  USING  FAILURE  DISCOUNTIN 
CG1  ) 

WRITE(30,3240' 

WRITE(30,3250i 

C   COMPUTE  C.I.  FOR  WEIGHTED  AVERAGE  ESTIMATES 

DO  3310  M=1,NPHASE 
CI  =  (1.96*EST(4,M))/SQRT(XNREP) 
CIU  =  EST(3,M)  +  CI 
CIL  =  EST(3,M)  -  CI 

WRITE (30, 3260)  M, AREL(M) ,EST(3 ,M) ,EST(4,M) , CIL, CIU 
3310  CONTINUE 

WRITE(30,3220) 
WRITE(30,3320) 
3320  FORMAT('0' ,T43,' REGRESSION  ESTIMATE  USING  DISCOUNTED  FAILURES') 
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WRITE(30,3240) 
WRITE(30,3250) 

C   COMPUTE  C.I.  FOR  EXPONENTIAL  REGRESSION  ESTIMATES 

DO  3330  M=1,NPHASE 

CI  =  (1.96*REGEST(4,M))/SQRT(XNREP) 
CIU  =  REGEST(3,M)  +  CI 
CIL  =  REGEST(3,M)  -  CI 

WRITE (30, 3260)  M,AREL(M) ,REGEST(3,M) ,REGEST(4,M) , CIL, CIU 
3330  CONTINUE 

WRITE (30, 3340) 
3340  FORMAT ( ' 1' ,T59. 'RECAPITULATION'//) 

WRITE(30,3350) 
3350  FORMAT( '-' ,T3, 'PHASE' ,T11, 'ACTUAL' ,T28, 'MEAN' ,T33, 'EST' ,T53, 'MEAN' 
C,T63, 'EST' ,T78, 'MEAN' ,T38, 'EST' ,T103, 'MEAN' ,T113, 'EST' ) 
WRITE(30,3360) 
3360  FORMAT( '  ' ,T11, 'RELIAB' ,T23, 'WGT' ,T38, 'STD' ,T53, 'MLE' ,T63, ' STD ' ,T7 
C7, 'PHASE' ,T88. 'STD' ,T103, 'REG' ,T113, 'STD' ) 
WRITE (30, 3370) 
3370  FORMAT( '  ' ,T28, 'AVG' ,T35, 'DEVIATION' ,T53, 'W/D' ,T60, 'DEVIATION' ,T78 
C, 'MLE' ,T85, 'DEVIATION' ,T103, 'EST' ,1110, 'DEVIATION' ) 
WRITE(30,3375) 
3375  FORMAT ('  ',T28,'EST'/) 
DO  650  I=1,NPHASE 

WRITE(30,3380)I,AREL(I) ,EST(3 , I) ,EST(4, I) ,MLEWD(3 , I) ,MLEWD(4, I ) , 
CMLES?(3,I) ,MLESP(4,I) ,REGEST(3 .1) ,REGEST(4,I) 
3380  FORMAT ( ' 0 ' , T4 , 13 , Til , F7 . 6 , T26 , F7 . 6 , T36 , F7 . 6 , T51 , F7 . 6 , T61 , F7 . 6 , T76 , 

CF7.6,T86,F7.6,T101,F7.6,T111,F7.6) 
650   CONTINUE 

STOP 

END 
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2.        SAMPLE  SIMULATION  INPUT  FILE 


NUMBER  OF  FAILURE  CAUSES 

NUMBER  OF  PHASES  (  NPHASE  ) 

NUMBER  OF  FAILURES  IN  PHASE  1 

NUMBER  OF  FAILURES  IN  PHASE  2 

NUMBER  OF  FAILURES  IN  PHASE  3 

NUMBER  OF  FAILURES  IN  PHASE  4 

NUMBER  OF  FAILURES  IN  PHASE  5 

NUMBER  OF  FAILURES  IN  PHASE  6 

NUMBER  OF  FAILURES  IN  PHASE  7 

NUMBER  OF  FAILURES  IN  PHASE  8 

NUMBER  OF  FAILURES  IN  PHASE  9 

NUMBER  OF  FAILURES  IN  PHASE  10 
.8  i  -  PROB.  OF  OCCURRENCE  FOR  CAUSE  1  IN  PHASE  1 

.8  1  -  PROB.  OF  OCCURRENCE  FOR  CAUSE  2  IN  PHASE  1 

.8  1  -  PROB.  OF  OCCURRENCE  FOR  CAUSE  3  IN  PHASE  1 

.7  1  -  PROB.  OF  OCCURRENCE  FOR  CAUSE  4  IN  PHASE  1 

.7  1  -  PROB.  OF  OCCURRENCE  FOR  CAUSE  5  IN  PHASE  1 

50  DISCOUNT  INTERVAL  FOR  STRAIGHT  PERCENT  DISCOUNTING 

.75  DISCOUNT  FRACTION  FOR  STRAIGHT  PERCENT  DISCOUNTING 

624712.0  RANDOM  NUMBER  SEED 

.95  RELIABILITY  GROWTH  FRACTION 

10000  NUMBER  OF  DESIRED  REPETITIONS  FOR  THE  SIMULATION 

0  INTERMEDIATE  OUTPUT  OPTION  (LINT.  OUTPUT;  0:  NO  OUTPUT) 

0  SAVE  ALL  WEIGHTED  AVERAGE  ESTIMATES  (1:  YES;  0:  NO  ) 

0  SAVE  ALL  MLE  W/  DISCOUNTING  ESTIMATES  (1:  YES;  0:  NO  ) 

0  SAVE  ALL  MLE  SINGLE  PHASE  ESTIMATES  (1:  YES;  0:  NO  ) 

0  SAVE  ALL  REGRESSION  ESTIMATES  (1:  YES;  0:  NO  ) 

2  DISCOUNTING  OPTION  (1:  STRAIGHT  %  ;  2:  LLOYD  METHOD) 

.2  GAMMA  FOR  LLOYD  DISCOUNTING  METHOD 
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3.       SAMPLE  SIMULATION  OUTPUT  FILE 


DISCRETE  RELIABILITY  SROWTH  SIMULATION 

HOOEL  PARAMETER  SUMMARY 
NUMBER  OF  POSSIBLE  FAILURE  CAUSES     5 


CAUSE  NUMBER 
1 

2 
3 

S 


SIN6LE  TRIAL  PRC  SUCCESS  3  FOR  PHASE  1 
0.800000 
0.800000 
0.300000 
0.700000 
,  0.700000 


FRACTION  CAUSE  RELIABILITY  IMPROVES  AFTER  FAILURE  0.950000 

NUMBER  OF  PHASES  IN  THE  SIMULATION  10 

PHASE  NUM8ER      NUMBER  OF  FAILURES  IN  THE  FIRST  PHASE 

1 
2 
3 
4 

5 
6 
7 

a 

9 

10 

TOTAL  NUMBER  OF  FAILURES    10 

DISCOUNTING  PERFORMED  USINS  THE  CONSTANT  FRACTION  METHOD 

FRACTION  EACH  FAILURE  IS  DISCOUNTED  0.750000 
NUMBER  OF  TRIALS  AFTER  A  FAILURE  BEFORE  A  OISCOUNT  IS  APPLIED    50 

RANDOM  NUMBER  SEEO  USEO        <24712.00 
NUMBER  OF  REPETITIONS  OF  THE  SIMULATION  PERFORMED      100 
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APPENDIX  B 
GRAPHICAL  RESULTS  OF  THE  RELIABILITY  GROWTH  MODEL  SIMULATION 

1.       MEAN  ESTIMATES  FOR  EACH  ESTIMATOR  WITH  CONSTANT  PARAMETERS 
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